-- Logs begin at Mon 2022-07-11 11:22:45 UTC, end at Mon 2022-07-11 12:14:00 UTC. -- Jul 11 11:52:26.620601 dv-cinder-849325-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 11 11:52:26.627847 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 11:52:26 2022] *** Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: nodename: dv-cinder-849325-1-iscsi Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: machine: x86_64 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: clock source: unix Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: pcre jit disabled Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: detected number of CPU cores: 4 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: current working directory: / Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: detected binary path: /usr/bin/uwsgi-core Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: your processes number limit is 47680 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: your memory page size is 4096 bytes Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: detected max file descriptor number: 2048 Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: lock engine: pthread robust mutexes Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: thunder lock: enabled Jul 11 11:52:26.631991 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 11 11:52:26.632862 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 11:52:26.679471 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: Python main interpreter initialized at 0x55de8c8ca9c0 Jul 11 11:52:26.679471 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: python threads support enabled Jul 11 11:52:26.679471 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: your server socket listen backlog is limited to 100 connections Jul 11 11:52:26.679471 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: your mercy for graceful operations on workers is 90 seconds Jul 11 11:52:26.679857 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 11:52:26.679933 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: *** Operational MODE: preforking *** Jul 11 11:52:26.680164 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: *** uWSGI is running in multiple interpreter mode *** Jul 11 11:52:26.680164 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: spawned uWSGI master process (pid: 80215) Jul 11 11:52:26.680373 dv-cinder-849325-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 11 11:52:26.681781 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: spawned uWSGI worker 1 (pid: 80222, cores: 1) Jul 11 11:52:26.681781 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: spawned uWSGI worker 2 (pid: 80223, cores: 1) Jul 11 11:52:26.681781 dv-cinder-849325-1-iscsi devstack@keystone.service[80215]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 11:52:32.131737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:52:32.131737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jul 11 11:52:32.132442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:52:32.132442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jul 11 11:52:32.133097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:52:32.133097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: from cryptography.utils import int_from_bytes Jul 11 11:52:32.133557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:52:32.133557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: from cryptography.utils import int_from_bytes Jul 11 11:52:32.180824 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 11:52:32.181007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 11:52:32.181209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] command line args: [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 11:52:32.181484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 11:52:32.181660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ================================================================================ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 11:52:32.181940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.182185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_dir = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.182373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.182500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 11:52:32.182640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_source = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.182734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 11:52:32.182820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] debug = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.182901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] command line args: [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 11:52:32.183250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 11:52:32.183319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ================================================================================ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 11:52:32.183452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] admin_token = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_dir = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.183983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_source = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_config_append = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] debug = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184493 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_dir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.184840 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185750 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.185995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_config_append = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_dir = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.186900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187112 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.187940 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188281 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.188992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] publish_errors = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189239 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.189897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] standard_threads = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] publish_errors = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.190935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_journal = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_json = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_stderr = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_syslog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.191833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.192470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] standard_threads = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.192551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.192665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.192771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.192914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.193052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.193173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.193298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.193418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_journal = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.193418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_json = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.193658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.193846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_stderr = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.193976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_syslog = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.194218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:52:32.194473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194589 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.194944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.195923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.196951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197775 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.197896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.198959 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199307 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.199899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200015 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200846 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.200961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201321 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.201975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.202932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.203949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204421 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204777 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.204900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205253 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205253 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205743 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.205978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206077 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.206924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207512 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.207996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.208989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.209945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.210924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211873 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.211989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.212952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.213891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.214943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.215980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.216836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217241 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.217883 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218241 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.external = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.218878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.password = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.token = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.219953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.external = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.220945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221064 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.password = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221810 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.221945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.token = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.222769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223738 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.223970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.224940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.225905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.226971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227561 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.227923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228403 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.228997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.229930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230607 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.230972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231689 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.231927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232422 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.232897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233869 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.233987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234350 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.234937 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.235997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.236924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237167 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237167 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.237906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.238026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.238144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.238261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.238384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.238502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.239979 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240635 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.240998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.241879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.242903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243149 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.243991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.244975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.245879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246235 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.246949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.247896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248375 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.248971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.249923 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250405 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.250882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251118 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251466 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.251948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.252891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.253932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254400 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.254996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.255873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.256939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.257908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258158 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.258898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259830 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.259948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.260903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.261908 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262510 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.262986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263341 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263699 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.263935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264295 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.264890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.265995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.266952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.267897 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268374 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.268977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269463 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269844 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.269964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.270186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:52:32.271145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 11:52:32.297532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.297782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.298011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.299725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.299957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.300189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.307999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.309875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.337036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.337263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.338787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.339025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.480374 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.480552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.528599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.528826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.529428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.529690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.594906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.595703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.620456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.620686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.620773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.620911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.620999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.621946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.622028 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.632989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.633309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.646338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.646563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.646784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.646868 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647514 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.647871 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.648089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:52:32.665519 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.665745 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:52:32.665976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.666084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.666191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:52:32.666283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:52:32.666529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.666612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.666732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:52:32.666823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.667771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.678150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.678425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.678689 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.678771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.678965 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.679059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.679259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.679543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.687218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.687899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.690510 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.690790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.691361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.691925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.692920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.693002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.693184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.693267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.693450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.693747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.694019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.720421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.721419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.722070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.722351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.723680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.723966 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.728456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.728750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.729179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.729442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.729770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.730202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.736867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.737089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.737798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.738107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.741817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.742118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.742399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.742675 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.742818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.742944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.743971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.744249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.744523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.744793 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.762716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.763011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.763955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.764256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.768015 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.769033 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.772402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.772707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.775050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.775380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.778008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.781736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.781913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.786086 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.786249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.786437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.786729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.787025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.787305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.787593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.789944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.790259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.790558 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.790853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.791143 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.791432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.807247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.807551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.807821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.808084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.810905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.811213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.811531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.811799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.816093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.817911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.818186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.819973 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.821405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.821710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.821818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.821995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.822079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.822275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.822620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.825302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.825590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.825900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.826185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.826466 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.836884 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.837175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.837458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.837773 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.838052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.839816 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.840115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.840761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.841063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.841545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.841649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.841926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.843749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.844048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.858795 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.862110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.862904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.865541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.866314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.867572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.868932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.869214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.870087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.870188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.871431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.873091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.873369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.873659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.876815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.878460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.881177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.881745 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.882031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.882424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.882709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.882846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.886138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.886429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.886824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.887102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.891832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.892119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.892395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.896862 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.897154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.897442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.900807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.901088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.901367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.901639 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.901930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.902308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.902589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.902863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.903137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.903405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.903672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.903940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.904210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.904471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.905364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:52:32.905646 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.905952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.906232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.906505 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.906895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.907175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.907454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.907736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.908009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.908298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.908565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.908832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.909100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:52:32.943747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55de8c8ca9c0 pid: 80222 (default app) Jul 11 11:52:32.947263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f626-4d9d-4a7f-a0e3-38320fba9b2d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:32.947557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f626-4d9d-4a7f-a0e3-38320fba9b2d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:32.947835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f626-4d9d-4a7f-a0e3-38320fba9b2d None None] PATH_INFO: `/v3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:32.948944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55de8c8ca9c0 pid: 80223 (default app) Jul 11 11:52:32.952018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Jul 11 11:52:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:40.919328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:40.919951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:40.919951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:40.924416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:40.931580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:40.932077 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:40.932568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:41.077374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: INFO dbcounter [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] Registered counter for database keystone Jul 11 11:52:41.114025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG oslo_db.sqlalchemy.engines [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 11:52:41.121606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writer thread running {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 11:52:41.307403 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80222) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 11:52:41.307893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80222) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 11:52:41.365054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:41.366770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:41.502522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 11:52:41.578123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:41.650109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: INFO dbcounter [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] Registered counter for database keystone Jul 11 11:52:41.670758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG oslo_db.sqlalchemy.engines [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80223) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 11:52:41.678238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writer thread running {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 11:52:41.804247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f183571-4293-4e6e-9f56-bb9bb2e1eb2b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:41.808812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:41.809506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:41.809826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:41.810110 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:41.819758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:41.874048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING keystone.common.rbac_enforcer.enforcer [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 11:52:41.874301 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee7f94-fbce-4147-b71a-3857ba5f0e2d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:41.886540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 11 11:52:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:52:41.893019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:41.909208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba3b06-adef-4225-a614-9ec521920daf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:41.914008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:41.915269 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:41.915608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:41.915937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] PATH_INFO: `/v3/projects/aeb90a4a47b148ccb3710178a3b17750` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:41.920719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] RBAC: Authorizing `identity:get_project(project_id=aeb90a4a47b148ccb3710178a3b17750)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:41.974654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 11:52:41.975354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fcab6-4e02-4981-8e0d-b4ddfa761de2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:41.980260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Jul 11 11:52:41 2022] GET /identity/v3/projects/aeb90a4a47b148ccb3710178a3b17750 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:44.085926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29be1-1e1a-4984-82e7-944b9021f9da None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:44.086441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29be1-1e1a-4984-82e7-944b9021f9da None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:44.086541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29be1-1e1a-4984-82e7-944b9021f9da None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:44.090617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:44.096360 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:44.096812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:44.096926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:44.122183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:44.123757 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:44.131802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 11:52:44.154797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29be1-1e1a-4984-82e7-944b9021f9da None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:44.167621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29be1-1e1a-4984-82e7-944b9021f9da None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:44.171749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:44.172389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:44.172674 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:44.172953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:44.180922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:44.182744 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5876-0822-4a2d-9a4e-6ac3c8873c28 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:44.192388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 11 11:52:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:52:44.198170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:44.208192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd82d3d-5f40-4b35-a96b-971c04e178b4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:44.213024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:44.213950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:44.214237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:44.214513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:44.215926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:44.218104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7474f-e3ba-40f4-9da4-9f18eec7610e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:44.219721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Jul 11 11:52:44 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:46.342615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f1ba-e72b-433a-b1f5-f86d9c5f1a5e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.343166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f1ba-e72b-433a-b1f5-f86d9c5f1a5e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.343166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f1ba-e72b-433a-b1f5-f86d9c5f1a5e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.344520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:46.349800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.350207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.350379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.367301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:46.368489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:46.441117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d9f5-00e2-4026-ad1e-25a71353743b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.441340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d9f5-00e2-4026-ad1e-25a71353743b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.442291 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d9f5-00e2-4026-ad1e-25a71353743b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.443684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:46.448759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.449056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.449343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.452963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:46.476671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.490477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG passlib.handlers.bcrypt [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80223) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 11:52:46.490897 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG passlib.handlers.bcrypt [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80223) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 11:52:46.502875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:46.536598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:46.538240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:46.580988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edd370-f7de-4bc1-8edb-7e05e1b46f0c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:46.615213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:46.616019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.616311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.616593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.627791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:52:46.637079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:52:46.639334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:52:46.640370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:52:46.641195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:52:46.646703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.658939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.740275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:46.749281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0949d-ce73-4834-a841-f86cdd75a612 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:46.751850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:46.760135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d06b3b-1261-48e9-b814-8861cdddfb50 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:46.788055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:46.788967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.789256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.789538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.790484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:46.790799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:46.792276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:46.794197 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:46.794856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.795149 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.795434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.804567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:46.810039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.864295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:46.865819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:46.908679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33371a-e78e-4c0f-ae7d-bad19eea9328 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:46.945803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:46.946452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:46.946685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:46.947245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:46.947245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:46.948813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:46.950509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:46.951947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:46.957847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.966610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:52:46.972088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:46.992224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:47.058272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7248bf-4978-4964-a03d-b551f59657ae None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:47.070867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e46e2-1873-4678-ade4-3be74344ec88 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:47.092702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:47.093518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:47.093860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:47.094147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:47.095354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:47.096816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:47.099103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:47.104941 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:47.105100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:47.105929 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:47.106217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:47.106495 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:47.116461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:47.117984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:52:47.137615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:52:47.138686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:52:47.139591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:52:47.139591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:52:47.139591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:52:47.139591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:52:47.139591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:52:47.145260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:47.179339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:47.211553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37b7aa-efe8-413c-810e-fe50ba60a54e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:47.213088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:47.249726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e077b-fcb2-4517-9d97-0d1c3a4e7bdf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:47.251450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:47.252119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:47.252412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:47.252698 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:47.253541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.domain.id=default, user.id=f4c463b509f1490f91e3e253190d2a42) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:47.254395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:47.255855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d07ec-6570-4081-9b57-82f81eaff769 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:47.264828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 11 11:52:47 2022] GET /identity/v3/role_assignments?user.id=f4c463b509f1490f91e3e253190d2a42&scope.domain.id=default&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:47.289877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:47.290570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:47.290860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:47.291139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:47.291892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:47.292206 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:47.293752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57561ddf-8f86-4a97-bb0e-11e07a5f225f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:47.313100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:49.672564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175bd6-6723-4e66-a27f-e18caf033f5f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:49.673192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175bd6-6723-4e66-a27f-e18caf033f5f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:49.673192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175bd6-6723-4e66-a27f-e18caf033f5f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:49.674502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:49.676140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36531-c715-447e-885c-7591da99f1ce None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:49.676573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36531-c715-447e-885c-7591da99f1ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:49.676753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36531-c715-447e-885c-7591da99f1ce None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:49.678104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 11/23] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:49.683714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:49.684010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:49.684294 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:49.685785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:49.686074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:49.686354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:49.716592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:49.723394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:49.734807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:49.736040 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:49.818722 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 12/24] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:49.843570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:49.872930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:49.908469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 13/25] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:52:49.938680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:49.957545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ba58d-73d5-4771-a3dd-b0178dd6d53c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.001015 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.001875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.002174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.002486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.016669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:50.026395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:52:50.054275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:52:50.055319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:52:50.056342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:52:50.056342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:52:50.056342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:52:50.056342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:52:50.058302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 13/26] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:52:50.064223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.155046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:50.183685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccdcce-db22-459a-88d9-01bc88054e1c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.234539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.235100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:50.235516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.235824 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.236380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.236858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:50.237178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:50.239379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:50.256007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46bcf7-e9a3-41fe-8418-a9508cf85fab None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.258187 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 14/27] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:50.263924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.324299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.325474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.325988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.326495 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.347060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7026fabc-9e64-45bc-91cb-01f284dfa61d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:52:50.360137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:52:50.361065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:52:50.361867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 14/28] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:52:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:52:50.368385 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b647-d9bc-4a35-bd19-9297494e83ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.368678 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b647-d9bc-4a35-bd19-9297494e83ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.369260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b647-d9bc-4a35-bd19-9297494e83ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.371257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 15/29] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:50.372551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2b2f-713d-48bd-9e79-6c30d9472a5c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.373041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2b2f-713d-48bd-9e79-6c30d9472a5c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.373486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2b2f-713d-48bd-9e79-6c30d9472a5c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.375262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 16/30] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:50.379934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba2b2f-713d-48bd-9e79-6c30d9472a5c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.416358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123192f-a597-47c8-a2ef-4836cc1d2301 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.454190 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.455034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.455333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.455679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.456488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:50.458330 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f816507-1899-49e1-91fe-7c46065e1f15 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:50.464142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:50.507330 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba2b2f-713d-48bd-9e79-6c30d9472a5c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.545788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.546476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.546794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.547048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.547831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:50.548145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:50.549544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59e6dd-f6b2-4f45-ac1e-cb9e25f0a23a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:50.560359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:50.562077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.562429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.562710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.584588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:50.585894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:50.605725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:50.624136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 15/32] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:52:50 2022] POST /identity/v3/projects => generated 301 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:52:50.625893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.626204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.626494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.658395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:50.659962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:50.681861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 18/33] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:50.682995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.850800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 16/34] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:50.852353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.868798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48322d-e0ba-4bf9-ab63-9ea7add0ea45 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:50.914127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:50.915036 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:50.915313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:50.915597 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:50.916909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:50.918752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:50.920444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 19/35] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:52:50 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:50.926702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:50.956186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:51.001798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487110b-4043-43d8-8eae-186857a1bc75 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:51.042979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:51.073777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d177c-4e77-49e7-941c-45fbdaf9dd1d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:51.080736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:51.082165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.082786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.083468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.085600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:51.088650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:51.125531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 17/36] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 11 11:52:50 2022] POST /identity/v3/roles => generated 225 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:52:51.126958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:51.141609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:51.143167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.143508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.143824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.145174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:51.146688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:51.151045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 20/37] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:51.157083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:51.257500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d8314-5379-405b-8a4a-587f9acd2e8e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:51.298061 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:51.299391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.299930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.300495 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.302148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:51.303790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf4249-04ed-4123-a4f3-a8586bf0a079 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:51.324138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 18/38] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 11 11:52:51 2022] POST /identity/v3/roles => generated 219 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:52:51.393212 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:51.414125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03035-8939-4e58-b8fe-9e356de72720 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:51.455144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:51.456432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.456927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.458044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] PATH_INFO: `/v3/domains/default/users/f4c463b509f1490f91e3e253190d2a42/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.461040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f4c463b509f1490f91e3e253190d2a42, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:51.463002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da19d2a-d98c-4fb8-b981-9b7004e2c23d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:51.598640 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 21/39] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Jul 11 11:52:51 2022] PUT /identity/v3/domains/default/users/f4c463b509f1490f91e3e253190d2a42/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:52:51.830242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd00-f732-4cf8-be8b-3e560fac1570 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.830753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd00-f732-4cf8-be8b-3e560fac1570 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.831245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd00-f732-4cf8-be8b-3e560fac1570 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.834585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 19/40] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:51.840767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3f2d-4b36-44b1-b2cb-7a4b45508f7a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:51.841476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3f2d-4b36-44b1-b2cb-7a4b45508f7a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:51.841476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3f2d-4b36-44b1-b2cb-7a4b45508f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:51.859977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e7df3f2d-4b36-44b1-b2cb-7a4b45508f7a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:51.861505 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e7df3f2d-4b36-44b1-b2cb-7a4b45508f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:51.939067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 22/41] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:51.962852 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8dd00-f732-4cf8-be8b-3e560fac1570 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:52.012845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823bd7-978c-47b0-9ef8-0d6269ba0257 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:52.013363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823bd7-978c-47b0-9ef8-0d6269ba0257 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:52.013918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823bd7-978c-47b0-9ef8-0d6269ba0257 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:52.023734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 23/42] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:52.030849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:52.031441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:52.031916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:52.062723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:52.064833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:52.119329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:52.187063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 24/43] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:52:52.208164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8dd00-f732-4cf8-be8b-3e560fac1570 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:52.211252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:52.214612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:52.268129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:52.269595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:52.270291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:52.271095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:52.272709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:52.276176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:52.300514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 20/44] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 11 11:52:51 2022] POST /identity/v3/roles => generated 223 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:52:52.363709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e9e8e-1712-427e-a2e2-8aef67c977da None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:52.411538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:52.412269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:52.412558 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:52.412901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:52.414377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:52.416054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0341b4b-2309-4c25-a150-6ca8995c566e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:52.418218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 25/45] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:52.423827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:52.564305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:52.569092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc78a01-de6b-41c5-977f-a05fdd21afb1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:52.627768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:52.628972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:52.629490 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:52.630036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:52.631383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:52.634201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff1051-907a-429d-a9f6-1d767df14d73 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:52.672650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 21/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:52:52 2022] POST /identity/v3/projects => generated 312 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:52:53.798466 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b41bc-1364-4169-b910-e59beb51b9a1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:53.799018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b41bc-1364-4169-b910-e59beb51b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:53.799018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b41bc-1364-4169-b910-e59beb51b9a1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:53.801788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 26/47] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:53.809503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:53.810024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:53.810335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:53.829875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:53.831122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:53.921788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 22/48] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:52:53.946273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b41bc-1364-4169-b910-e59beb51b9a1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:54.093185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b41bc-1364-4169-b910-e59beb51b9a1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:54.139934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:54.140770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.141062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.141346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.155192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:52:54.163890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:52:54.165575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:52:54.167121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 27/49] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:52:54.172219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.310980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:54.335140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7fe5d-8722-46d8-8845-fefb1c7fc050 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:54.375520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:54.376180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.376536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.377036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.377971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:54.378354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:54.379875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:54.397902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 23/50] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:52:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:54.406653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.526046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a2a5-0680-4d93-aab6-206c813f02e8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:54.560552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:54.568594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:54.570226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.570583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.570872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.572812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:54.575529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe95236-7436-45d5-80c7-7b24d11c1527 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:54.577404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 28/51] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:52:54 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:54.587235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.768395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74a59-25fc-4f86-bd44-6ec32f0972dd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:54.789440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ef6bb-2909-4502-91c0-5e63906e53b4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.789999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ef6bb-2909-4502-91c0-5e63906e53b4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.790566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ef6bb-2909-4502-91c0-5e63906e53b4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.793280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 29/52] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:52:54.802503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.803526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.803526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.818674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:54.819624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.819924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.820268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.822103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:54.823634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:54.825366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 24/53] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:52:54.831305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.833324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:54.835499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:54.846848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:54.939844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23040-a15e-43b5-bd50-5af491bfb5f1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:54.953020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 30/54] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:52:54.977931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:54.978222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:54.978789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:54.979080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:54.979365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:54.980219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.domain.id=default, user.id=f4c463b509f1490f91e3e253190d2a42) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:52:54.980538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:54.982054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:54.989737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 25/55] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 11 11:52:54 2022] GET /identity/v3/role_assignments?user.id=f4c463b509f1490f91e3e253190d2a42&scope.domain.id=default&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:55.159525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696f695-f5da-4d70-880c-697b18db1843 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:55.199313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:55.207612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:55.208591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:55.209025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:55.209718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:55.211488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:55.213373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4d34-59cc-4c6e-9ccc-952a3ae6b3db None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:55.215638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 31/56] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:55.222650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:55.314733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:55.343901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c741012-7111-44ba-b64e-c47dbd07f281 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:55.382208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:55.382805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:55.383100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:55.383387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:55.384133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:55.385615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc5dc9-d7c4-432d-add5-83bd7692dd27 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:55.414135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 26/57] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 11 11:52:55 2022] POST /identity/v3/projects => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:52:57.552063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566689-d0b3-4419-af67-07835f07e38d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:57.552659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566689-d0b3-4419-af67-07835f07e38d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:57.552659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566689-d0b3-4419-af67-07835f07e38d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:57.554166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 32/58] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:52:57.560598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:57.561188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:57.561349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:57.579661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:52:57.580914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:52:57.652639 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 27/59] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:52:57.678331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566689-d0b3-4419-af67-07835f07e38d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:57.896315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566689-d0b3-4419-af67-07835f07e38d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:57.931427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:57.932238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:57.932531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:57.932817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:57.934068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:57.935560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aad956-06da-43ec-964f-d8075ff53879 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:57.937242 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 33/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:52:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:52:57.944245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:52:58.011677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:52:58.057804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d1856-eb54-4fca-bd66-87f15569f1de None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:52:58.095868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:52:58.096565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:52:58.096861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:52:58.097153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:52:58.097991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:52:58.099542 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cfd5-e2d0-49c2-a611-faf3c13b725c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:52:58.133617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 28/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:52:57 2022] POST /identity/v3/users => generated 275 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:01.804768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162796c-6ce4-4fa7-8021-73af639b56a6 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:01.805704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162796c-6ce4-4fa7-8021-73af639b56a6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:01.806143 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162796c-6ce4-4fa7-8021-73af639b56a6 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:01.807766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 34/62] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:01.813868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:01.814850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:01.815344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:01.841132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:01.842855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:01.948164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 29/63] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:01.988737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162796c-6ce4-4fa7-8021-73af639b56a6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.134089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:02.203325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162796c-6ce4-4fa7-8021-73af639b56a6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:02.266668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:02.267535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.267836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.268124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.287092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b8cb3c51-a497-4275-8ee1-bb561b9b941b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:02.295397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:02.296369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:02.297150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 35/64] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:02.303392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.341879 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8f168-9c8c-433b-9b91-767c9c3cf92a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.342174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8f168-9c8c-433b-9b91-767c9c3cf92a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.342633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8f168-9c8c-433b-9b91-767c9c3cf92a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.344299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 36/65] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:02.350373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151d09-9031-47ff-bd84-b4d57358cbee None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.350665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151d09-9031-47ff-bd84-b4d57358cbee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.350950 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151d09-9031-47ff-bd84-b4d57358cbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.384576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-26151d09-9031-47ff-bd84-b4d57358cbee None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:02.385844 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-26151d09-9031-47ff-bd84-b4d57358cbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:02.398771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:02.481196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 37/66] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:02.482822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93affa6e-033f-4eaf-af5f-889798e99168 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:02.484654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80040-1079-4be4-a06e-001a7e22b3f3 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.485551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80040-1079-4be4-a06e-001a7e22b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.486400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80040-1079-4be4-a06e-001a7e22b3f3 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.490302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 38/67] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:02.500293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.501237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.502055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.535412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:02.536111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.536402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.536692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.537473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:02.537830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:02.539284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:02.542169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:02.544771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:02.549752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 30/68] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:02.550688 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.626177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:02.645803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 39/69] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:02.646583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.702443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c04c-97b8-42ee-ab23-92cf2e9b031e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:02.705949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:02.741980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:02.742809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.743114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.743386 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.755422 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c1ab9446-f0cb-46ea-82a9-909f3c9d69ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:02.764574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:02.765659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:02.766591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 31/70] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:02.767232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c852f5-812d-4cc9-b11a-e16a692a6828 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.767451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c852f5-812d-4cc9-b11a-e16a692a6828 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.767733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c852f5-812d-4cc9-b11a-e16a692a6828 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.769096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 32/71] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:02.773925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c852f5-812d-4cc9-b11a-e16a692a6828 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.825549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef07ce-9e47-4e07-94eb-59361a22f0a3 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:02.871037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:02.872333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.872828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.873319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.895305 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:02.896943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:02.898754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 40/72] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:02 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:02.899658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:02.910539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c852f5-812d-4cc9-b11a-e16a692a6828 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:02.953844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:02.954767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.955127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.955490 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:02.960326 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:02.968328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7fa91032-8618-4a0a-9cce-ab107e022b78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:02.976967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:02.978573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 33/73] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:02.980119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:02.981434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:02.981434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.010945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:03.013049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:03.035271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bd041-8a5a-402b-9a9f-00fb2bdda71a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.044075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.089507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.090248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.090545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.090827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.091681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:03.092132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.093317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.110043 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 41/74] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:03.110043 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.120707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 34/75] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:03.126144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.247893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade5565-dee9-46a8-ae0e-2a5031ed95a0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.256845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.265727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6c2c-dcd9-4442-8bf9-d4fd983c36f5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.297002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.298197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.298721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.299228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.308659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.309343 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.309636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.309958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.310730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:03.311048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.312471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.322500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 35/76] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:03.323376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.344584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.387995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.389471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea82fef-b047-4f45-988e-5337aa702474 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.485454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b17e9f-3812-4a15-9b75-03653f27348e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.513316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 42/77] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:03.513316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46448a69-0141-4302-a91f-b0dad1ed5f7f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.513316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46448a69-0141-4302-a91f-b0dad1ed5f7f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.513316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46448a69-0141-4302-a91f-b0dad1ed5f7f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.513316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 43/78] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:03.513987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46448a69-0141-4302-a91f-b0dad1ed5f7f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.531733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.536011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.536852 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.537139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.537492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.538822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.540310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.542018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 36/79] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:03.542811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.640644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.660584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef421f2a-4b0a-4731-8c1c-7d1a3ba566a6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.687097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46448a69-0141-4302-a91f-b0dad1ed5f7f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.698731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.699638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.699926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.700202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.701502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.702979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.704647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 37/80] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:03.705436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.741061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.741896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.742190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.742466 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.754847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b7e0dbd8-29c6-4061-bfea-99182eb688bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:03.762839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:03.764576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 44/81] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:03.766120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.768994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.768994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.787424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:03.788657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:03.837915 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf00f0-c879-40b5-8902-5f92d472c3ca None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:03.864426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:03.877906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 45/82] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:03.879435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.891877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:03.893029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:03.893464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:03.893915 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:03.896178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:03.897928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:03.903219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 38/83] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:03.903219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:03.932780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.005347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931601b9-2358-47e0-a8d9-748c4e167420 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.019036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128869f-1d5a-48d2-b017-7b9123cd4fc0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.043716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.044614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.044989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.045467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.056692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.057354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.057646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.057963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.111412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.119762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.121357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.135993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.137512 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.159735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.241484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 46/84] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:04.242500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.261328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 39/85] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:04.262228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.340192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.346734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff1f03-3028-4ecc-baa0-778f88927035 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.374233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c142-12a1-41a1-b145-bb24dda8cee7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.383447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.384134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.384426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.384754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.385496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:04.385848 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.387267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.398568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 47/86] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.399779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.399948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.422323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.423607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.424047 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.424491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.426539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.429096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.436334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 40/87] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.436334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.524122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eecc33-875b-4e7e-ade7-09af791de605 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.542161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e00de-9546-404a-b9dc-958102d2ec8e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.561801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.562470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.562759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.563042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.563885 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:04.564197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.565711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.572205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 48/88] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.573053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.593819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.594634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.594927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.595209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.596614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.598441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.607821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 41/89] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.607821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.613433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.682889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.729785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125ea1-f452-4566-b701-4fb7ab72ef0f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.743518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5af0-a236-43bf-89fd-85a8d5e9dbb0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.774057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.774952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.775245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.775532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] PATH_INFO: `/v3/projects/1fe219c085b743eaae0506e97f8ba868` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.776864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] RBAC: Authorizing `identity:get_project(project_id=1fe219c085b743eaae0506e97f8ba868)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.778381 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.780184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 49/90] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/projects/1fe219c085b743eaae0506e97f8ba868 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.780961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.799957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.801555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.802082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.802560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.804380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.806742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.809128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 42/91] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:04.811489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:04.917436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:04.933808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88ecc9-e6c7-4301-9831-c6fef14810b6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:04.975911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:04.976851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:04.977263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:04.978202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:04.978847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:04.980684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:04.989001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9d55-9476-422a-89c3-0857cedfe1fd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:05.015244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 50/92] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:04 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:05.016587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:05.049038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:05.074983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:05.076388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:05.076883 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:05.077364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:05.078971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:05.079513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:05.083559 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:05.101716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 43/93] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:04 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:05.101716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:05.289777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0512e4-5d99-4a67-95c5-d08f8c638db0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:05.368964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:05.370313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:05.370933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:05.371411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:05.372942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=1fe219c085b743eaae0506e97f8ba868, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:05.373704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:05.376842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:05.387954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 51/94] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:05 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=1fe219c085b743eaae0506e97f8ba868&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:05.412262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec589a-cf50-4b48-aea3-342514818836 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:05.485130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:05.486324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:05.486837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:05.487341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:05.606071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:05.634144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:05.636767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:05.845931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 44/95] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:05.852421 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:05.879060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:06.028689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a949a46-8e95-4727-847f-b778630a51dd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:06.083091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:06.084630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:06.085102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:06.085581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:06.087213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:06.088957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ebd38-74a0-417a-8c24-efce422c563e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:06.091457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 52/96] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:05 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:06.100925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:06.299680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:06.318098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66a2f-3e5f-4a07-aa3b-c6a43006ad7c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:06.355656 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:06.356333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:06.356619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:06.356904 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:06.357789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=f4c463b509f1490f91e3e253190d2a42) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:06.358112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:06.359593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715f7ca-e312-4f00-b009-85ac4c4e0a64 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:06.366769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 45/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:06 2022] GET /identity/v3/role_assignments?user.id=f4c463b509f1490f91e3e253190d2a42&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:07.485343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284ce03-fc3b-4f10-b593-0e7c68b0210e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:07.485891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284ce03-fc3b-4f10-b593-0e7c68b0210e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:07.485891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284ce03-fc3b-4f10-b593-0e7c68b0210e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:07.487238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 53/98] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:07.494320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:07.494835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:07.495409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:07.529263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:07.531402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:07.665231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 46/99] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:07.702273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284ce03-fc3b-4f10-b593-0e7c68b0210e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:07.926742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:07.982899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284ce03-fc3b-4f10-b593-0e7c68b0210e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:08.070939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:08.072393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:08.072915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:08.073433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:08.101430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:08.117993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:08.119167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:08.120080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:08.120711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 54/100] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:08.136004 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:08.366336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:08.402063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924975-9146-42db-9830-e4fc24878b71 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:08.492629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:08.492629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:08.492629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:08.492629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:08.493050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:08.493050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:08.493050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:08.505224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 47/101] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:08.511611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:08.745988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc61f2-d7bc-4ea6-94eb-a1040c2808ef None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:08.790218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:08.791539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:08.792047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:08.792538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:08.794148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:08.795928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:08.798293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 55/102] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:08 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:08.806826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:08.948808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6053f-c7f4-417b-9522-7f468358ec93 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:09.001222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:09.002135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.002432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.002717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.004180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:09.005720 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a653a5f-5e2d-4a99-a742-3548434ec96f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:09.007963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 48/103] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:08 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:09.014554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:09.027364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:09.147293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afaca1-5137-4369-a77f-cfc124b81a81 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:09.159346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7571-5dea-4c94-bbea-16120529df1f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.160839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7571-5dea-4c94-bbea-16120529df1f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.161428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a7571-5dea-4c94-bbea-16120529df1f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.163168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 49/104] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:09.171241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93607e-7b12-4e00-b988-fec48ed2e574 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.171879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93607e-7b12-4e00-b988-fec48ed2e574 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.172372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93607e-7b12-4e00-b988-fec48ed2e574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.197376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:09.198340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.198634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.198913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/d41522a9f214417fad9710b6bfce9f18/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.208559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, user_id=d41522a9f214417fad9710b6bfce9f18, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:09.208848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eb93607e-7b12-4e00-b988-fec48ed2e574 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:09.210646 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:09.210936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb93607e-7b12-4e00-b988-fec48ed2e574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:09.237581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:09.248436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 56/105] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:09 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/d41522a9f214417fad9710b6bfce9f18/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:09.315902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 50/106] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:09.338739 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:09.388291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e334dd-4494-4b00-af3c-44614c3546f4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.389106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e334dd-4494-4b00-af3c-44614c3546f4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.389879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e334dd-4494-4b00-af3c-44614c3546f4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.392644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 51/107] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:09.400772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a4bd-6f69-45e2-85c0-0ef6927c0f77 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.401571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a4bd-6f69-45e2-85c0-0ef6927c0f77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.402339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a4bd-6f69-45e2-85c0-0ef6927c0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.451742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d614a4bd-6f69-45e2-85c0-0ef6927c0f77 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:09.455748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d614a4bd-6f69-45e2-85c0-0ef6927c0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:09.460089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:09.581016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109031-0761-4a77-9386-4e5c4f526b0e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:09.670891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 52/108] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:09.672475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773aa78-5f1d-4005-9031-3a6fbb5d9d00 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.672780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773aa78-5f1d-4005-9031-3a6fbb5d9d00 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.673199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773aa78-5f1d-4005-9031-3a6fbb5d9d00 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.674619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 53/109] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:09.680305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.680580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.680874 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.682307 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:09.683098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.683480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.683774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.696884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:09.705106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-741acb80-cd0b-4527-ab75-47e9356e83d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:09.705440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:09.706427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:09.707799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 57/110] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:09.708464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:09.709125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f819e-7d9b-484b-b0b2-9b66f3709492 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.709125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f819e-7d9b-484b-b0b2-9b66f3709492 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.709607 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f819e-7d9b-484b-b0b2-9b66f3709492 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.711225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 58/111] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:09.714651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f819e-7d9b-484b-b0b2-9b66f3709492 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:09.810174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 54/112] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:09.811374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:09.814654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:09.849198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f819e-7d9b-484b-b0b2-9b66f3709492 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:09.878187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:09.890046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:09.890948 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.891225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.891441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.913492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c5fedcd6-c162-4d13-b4cf-c0e1b4347e58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:09.929104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:09.930073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:09.931211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:09.932023 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 59/113] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:09.933660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:09.934216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:09.934743 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:09.963403 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:09.964655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:09.971681 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d79fa-c6a7-490d-81de-1aa7012d7aff None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.010151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.011098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.011391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.011670 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.012449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:10.012761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.014238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c64c-cdc6-453d-b1e1-1ff149fce558 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.025632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 55/114] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.031307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205044-a7df-414c-b4d7-06c1a1497d86 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.031615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205044-a7df-414c-b4d7-06c1a1497d86 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.031909 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205044-a7df-414c-b4d7-06c1a1497d86 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.033311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 56/115] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:10.034101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85205044-a7df-414c-b4d7-06c1a1497d86 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.051000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 60/116] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:10.053450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.136852 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.188330 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85205044-a7df-414c-b4d7-06c1a1497d86 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.202859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b8ed9-189c-4b33-a38b-2d06e3649e80 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.224653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.235257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.236099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.236399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.236687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.249819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:10.250855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.252078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.252615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.253230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.254804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:10.255369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.257761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7ee22-f276-45a3-8f96-0d90d076eeac None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:10.258013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:10.258868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:10.259637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 57/117] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:10.260294 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.277964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 61/118] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.277964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.277964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.277964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.303225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:10.304897 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:10.405966 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 62/119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:10.407672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5259f-95d4-4645-a52f-28032ea22257 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.407823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.449831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.464443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.465339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.465627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.465949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.467184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.468664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.470680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 58/120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.475977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.563160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb203b-3e18-4539-a91e-5d7725240769 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.580625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.621898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4181efa-4348-42f9-a293-9e519a72f83a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.632987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.635190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.636122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.636611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.639031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.641262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.649715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 63/121] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.649715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.679953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.680737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.680881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.681119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.681927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:10.682241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.683674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.694298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 59/122] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.701727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.796818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.821651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0854-6c9d-4b32-ab96-8b9a283d7fb3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.852622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c9e-602b-4921-8794-bbf75897c4a7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:10.880485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.881504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.881881 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.882246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.883701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:10.885232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:10.886998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 64/123] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:10.887813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:10.918963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:10.920283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:10.920775 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:10.921233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:10.940296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:10.952472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:10.956464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:10.957444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:10.958273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:10.958887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 60/124] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:10.962804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.061175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb53de-c165-4abe-8412-235a90c72338 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.138601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.140165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.140673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.141189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.143650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.144550 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9f5e6-b822-4371-a0c1-2fdc339c8afd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.146603 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.153949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 65/125] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:10 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:11.153949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.181593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:11.226599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.227542 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.228135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.228613 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.229964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.232680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.279873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 61/126] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:53:10 2022] POST /identity/v3/projects => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:11.281813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.322791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b87f0-6bf4-46f4-aa6c-5b80533f7655 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.361801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:11.414625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.416530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.417308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.418028 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.420595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.423643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.433367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 66/127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:11 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:11.435472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.547551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc58cd-a92b-4d6f-9f71-52daef1aac5b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.626790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.628815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.629360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b96201-46d2-474a-b383-2aefbd2d71d2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.629991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.630723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] PATH_INFO: `/v3/projects/1fe219c085b743eaae0506e97f8ba868` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.633257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] RBAC: Authorizing `identity:get_project(project_id=1fe219c085b743eaae0506e97f8ba868)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.636564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.640220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 62/128] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:11 2022] GET /identity/v3/projects/1fe219c085b743eaae0506e97f8ba868 => generated 312 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:11.642154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.694222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:11.705554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.707246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.707775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.708147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.709760 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:11.710785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.713192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.735580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 67/129] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:11.737257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.778509 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498a9fb-e003-4fdd-a411-73f678109292 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.814662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:11.817025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:11.817859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:11.818220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:11.818444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/d41522a9f214417fad9710b6bfce9f18/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:11.820865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, user_id=d41522a9f214417fad9710b6bfce9f18, role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:11.822405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:11.845714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 63/130] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:11 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/d41522a9f214417fad9710b6bfce9f18/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:11.845714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:11.940203 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4c60-33ac-459f-bdcd-88d0a8f20d87 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:11.980124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:11.991108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e16dc-3405-45dc-a256-fff28a830e54 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:12.013939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.015463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.015973 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.016484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.019109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:12.022687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533370f7-a06b-4a12-8106-ccb893639c0b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:12.026450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 68/131] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:11 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:12.029128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ab00-0930-4c13-89dc-b63d415eb3f4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.029666 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ab00-0930-4c13-89dc-b63d415eb3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.030516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ab00-0930-4c13-89dc-b63d415eb3f4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.033750 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 69/132] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:12.035714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2ab00-0930-4c13-89dc-b63d415eb3f4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:12.044391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.045707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.046207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.046690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] PATH_INFO: `/v3/projects/1fe219c085b743eaae0506e97f8ba868/users/d41522a9f214417fad9710b6bfce9f18/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.049229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fe219c085b743eaae0506e97f8ba868, user_id=d41522a9f214417fad9710b6bfce9f18, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:12.051098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:12.069245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 64/133] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:11 2022] PUT /identity/v3/projects/1fe219c085b743eaae0506e97f8ba868/users/d41522a9f214417fad9710b6bfce9f18/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:12.070459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:12.205029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2ab00-0930-4c13-89dc-b63d415eb3f4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:12.219332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:12.248811 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.250221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.250737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.251274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] PATH_INFO: `/v3/users/eea1a45cc08d4b8da5906ae469514256` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.272378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] RBAC: Authorizing `identity:get_user(user_id=eea1a45cc08d4b8da5906ae469514256)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:12.272711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a87a-ce65-4938-8a00-4b8c60b2c1f3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:12.274480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11156b-f270-4487-b798-89dc47ca5685 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:12.276791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 70/134] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:12 2022] GET /identity/v3/users/eea1a45cc08d4b8da5906ae469514256 => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:12.278498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.280088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.280595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.299965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:12.302131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:12.351127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.352725 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.353238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.354144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/f4c463b509f1490f91e3e253190d2a42/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.357547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, user_id=f4c463b509f1490f91e3e253190d2a42, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:12.360315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:12.387857 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 71/135] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:12.389149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:12.389574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 65/136] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:12 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/f4c463b509f1490f91e3e253190d2a42/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:12.428406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:12.483463 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:12.516340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:12.532382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f59f6-e5e4-4c6a-b831-aa02f993ebe5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:12.618864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.620195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.621011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.621727 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.723214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a759-fc16-49f6-a9b8-42ad23bb9b5c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:12.778989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:12.781275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:12.801521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:12.803655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:12.804243 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:12.805045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:12.832316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:12.855793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:12.856731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:12.857529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:12.858186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 66/137] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:12.867766 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:12.923106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:12.934314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 72/138] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:12.936524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:13.081224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:13.163055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579322a6-d666-423a-8def-fee09822b2e9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:13.189282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4127-3c88-452f-9131-f6df0c4c370e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:13.232637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:13.233824 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:13.234337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:13.234836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:13.236247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:13.236802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:13.239955 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:13.256383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 67/139] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:13.266468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:13.298446 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:13.299796 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:13.300357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:13.300846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:13.302468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:13.304193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:13.306239 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 73/140] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:12 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:13.312533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:13.476854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726849-46c4-4ea1-91d8-4f4a9fd99f4f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:13.512626 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:13.540517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:13.542508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:13.543245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:13.543944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:13.546413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:13.549581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:13.552789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 68/141] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:13 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:13.563100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:13.572726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:13.580332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526d978-61d7-4926-bdd5-7f5e0aa00e5a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:13.694418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:13.695105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:13.695401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:13.696880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:13.697832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=eea1a45cc08d4b8da5906ae469514256) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:13.698163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:13.699670 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:13.716989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 74/142] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:13 2022] GET /identity/v3/role_assignments?user.id=eea1a45cc08d4b8da5906ae469514256&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 259 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:13.849134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c30e64-d91d-4eca-9a22-65d98c61898e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:13.939649 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:13.940800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:13.941936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:13.942425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:14.053662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:14.056304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:14.070530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:14.205518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 69/143] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:53:14.214068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:14.359439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a229-daca-45a6-b80d-32b7e11d02c5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:14.400395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:14.406813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:14.407761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:14.408043 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:14.408290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:14.409642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:14.411209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ad04-11e3-46af-88b7-5d667ed5752b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:14.415723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 75/144] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:14 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:14.424774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:14.556589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee31b47-e75c-475f-83a1-149341db4776 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:14.561621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:14.610999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:14.611743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:14.612033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:14.612302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:14.613787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:14.614110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:14.615599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04308e-0a3c-4f7c-aa88-e0d2c309ccd4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:14.627307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 70/145] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:14 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:15.266335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8601-488f-463a-b768-7633cb5a6f0a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.267066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8601-488f-463a-b768-7633cb5a6f0a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.267066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8601-488f-463a-b768-7633cb5a6f0a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.268668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 76/146] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:15.275275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.275986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.275986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.296594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:15.297851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:15.409700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 71/147] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:15.437977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8601-488f-463a-b768-7633cb5a6f0a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:15.583932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8601-488f-463a-b768-7633cb5a6f0a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:15.627004 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:15.627829 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.628116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.628434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.632271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:15.633814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5757ee-56b0-409a-b743-2609bb583d54 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:15.635583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 77/148] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:15.641911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:15.699491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a066-2405-4c76-8bf7-89c464ebd48e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.699704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a066-2405-4c76-8bf7-89c464ebd48e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.699995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a066-2405-4c76-8bf7-89c464ebd48e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.701430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 78/149] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:15.708010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331ad16-cc21-425d-b014-effdb699abaf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.708436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331ad16-cc21-425d-b014-effdb699abaf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.708812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331ad16-cc21-425d-b014-effdb699abaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.728232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a331ad16-cc21-425d-b014-effdb699abaf None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:15.729565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a331ad16-cc21-425d-b014-effdb699abaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:15.802171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:15.807634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 79/150] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:15.814179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d797c7c-d4de-4210-b4f5-5c5994c95e23 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.814661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d797c7c-d4de-4210-b4f5-5c5994c95e23 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.815421 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d797c7c-d4de-4210-b4f5-5c5994c95e23 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.822217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 80/151] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:15.824441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.824815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.825104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.853807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:15.855084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:15.870199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:15.874985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1c26-0924-4b5e-bc3f-0a6133175e83 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:15.917262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:15.918112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:15.918473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:15.918817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:15.919742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:15.921364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:15.938926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 81/152] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:15.939649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:15.954878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 72/153] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:15 2022] POST /identity/v3/users => generated 283 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:15.974859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.061958 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1228baa-afcb-4af7-a725-c566c6750e45 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.099844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d369-9ba0-4d75-a8e3-0647b0bccaf6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.104348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:16.120386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.121247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.121534 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.121840 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.145701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.147392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.147881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.148312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.165221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:16.175124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:16.182952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:16.184925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:16.186717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:16.190118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:16.192496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:16.194533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 73/154] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:16.198212 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 82/155] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:16.198787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.205023 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.250214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:16.379800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a1c4-0920-4528-895d-4d967daed98f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.418917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9ff82-b983-4acb-81fb-eaa62825fcf1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.449769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.451219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.452057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.453156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.454981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:16.455920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:16.460491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:16.462679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.463379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.463672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.463962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.464746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:16.465109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:16.466750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:16.478108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 74/156] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:16.481244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 83/157] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:16.483819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.489709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.562227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:16.595619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:16.619894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f04d38-36db-4586-9b5d-36e6225fdfc0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.668383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a65d-e250-418c-9595-ec2267a0dab0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.694938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.694938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.694938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.694938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] PATH_INFO: `/v3/users/f4c463b509f1490f91e3e253190d2a42` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.697840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] RBAC: Authorizing `identity:get_user(user_id=f4c463b509f1490f91e3e253190d2a42)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:16.701073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee067723-da65-47d1-b41f-d44f636b0c10 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:16.704681 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 75/158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/users/f4c463b509f1490f91e3e253190d2a42 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:16.707545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958908a-d35f-4aab-bbbb-e951d43d0903 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.708335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958908a-d35f-4aab-bbbb-e951d43d0903 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.709038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958908a-d35f-4aab-bbbb-e951d43d0903 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.714517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 76/159] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:16.719829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958908a-d35f-4aab-bbbb-e951d43d0903 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.764663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.766658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.767403 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.768781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.772747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:16.777482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82c1c6-8320-4b7e-a99d-f0d1a567f568 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:16.781797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 84/160] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:16.786373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.787182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.787911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:16.825576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:16.829469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:16.898910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958908a-d35f-4aab-bbbb-e951d43d0903 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:16.961263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 85/161] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:16.962091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:16.984927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:16.986796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:16.987456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:16.988097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:17.024092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:17.040601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:17.127318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43ef12-f95c-49e1-9284-4fa4a74f4574 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:17.181505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:17.181999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:17.182681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:17.182971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:17.183249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:17.184774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:17.337725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:17.342933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b209117-e2a5-4da0-9e72-1dbf78239364 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:17.422059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 77/162] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:17.424489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:17.447101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:17.566474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 86/163] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:17.568732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db780c-5fda-4713-a78b-fed3fd3ccb9f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:17.569253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db780c-5fda-4713-a78b-fed3fd3ccb9f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:17.569789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db780c-5fda-4713-a78b-fed3fd3ccb9f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:17.572344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 87/164] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:17.574695 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db780c-5fda-4713-a78b-fed3fd3ccb9f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:17.592260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:17.671127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0c1ce-1631-4c2c-9fbd-a17cd2af1643 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:17.734690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:17.735517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:17.735804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:17.736085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:17.751743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c6e82cdf-230e-4fd9-8b6b-9654c3c84d14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:17.761478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:17.763141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 78/165] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:17.764660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:17.768014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:17.768014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:17.785743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:17.786927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:17.898233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 79/166] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:17.905707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:17.913479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db780c-5fda-4713-a78b-fed3fd3ccb9f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.007499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.008392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.008672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.008943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.010071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:18.012483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:18.014060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:18.016710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 88/167] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:17 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:18.017047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.072450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:18.183115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b755b18-1ead-4935-a25f-d8f509e927bf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.189103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6727d-5f50-4b8f-93f6-ba682472498d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.236518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.237587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.238201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.238681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.239659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:18.240316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:18.242011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:18.271235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.272185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 89/168] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:18.272701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.273052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.273168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.273644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.283100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:18.286058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:18.289426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 80/169] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:17 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:18.290624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.480136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa74996-9bc4-47d3-9f10-d4a1aa067d45 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.484338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:18.526061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:18.542286 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.543415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.543912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.544412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.546059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=f4c463b509f1490f91e3e253190d2a42) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:18.546623 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:18.549408 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:18.561222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 81/170] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/role_assignments?user.id=f4c463b509f1490f91e3e253190d2a42&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:18.562761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.582899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c162b0-da2d-46b8-a084-991b865f779a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.711415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc752d8e-a8af-4c7d-8b59-d6b97a93ffbf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.726546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.727381 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.727672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.727957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.774925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:18.779470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.780709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.781246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.781845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] PATH_INFO: `/v3/users/d41522a9f214417fad9710b6bfce9f18` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.783380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] RBAC: Authorizing `identity:get_user(user_id=d41522a9f214417fad9710b6bfce9f18)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:18.785636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:18.791219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:18.792165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 90/172] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:18.793520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.794786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 82/172] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/users/d41522a9f214417fad9710b6bfce9f18 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:18.802468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:18.866801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:18.947461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e4208-0c92-4390-ad9f-e8bcd8b0af85 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:18.992625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:18.993745 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:18.994244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:18.995447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:18.996608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:18.998737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5ce50-6c2f-445f-abbf-5b1b84a31e37 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.002169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.009729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.034278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 83/173] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:19.036352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.114570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.115716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.116228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.116622 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.117799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:19.118380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.121222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.130081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:19.145620 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 91/174] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:18 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:19.146689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.280344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c184-6b3e-4146-937e-20c63be25299 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.285188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:19.338206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3344ae4-0fee-470c-ba7a-7d214eb64abb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.338533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.339169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.339465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.339753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.377052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.377988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.379369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.379650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] PATH_INFO: `/v3/users/eea1a45cc08d4b8da5906ae469514256` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.381124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] RBAC: Authorizing `identity:get_user(user_id=eea1a45cc08d4b8da5906ae469514256)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.382697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.384706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 92/175] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/users/eea1a45cc08d4b8da5906ae469514256 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:19.394996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.491587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.493102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.495566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:19.567942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105f1d8-a021-44b7-8c44-6368484f02d6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.608833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.609754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.610044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.610327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.611687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.613823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867917d-0704-465e-a4ce-9be838508f2b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.615731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 93/176] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:19.617075 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d05ad-b275-4967-a1c4-d4bae2c21761 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.617366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d05ad-b275-4967-a1c4-d4bae2c21761 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.617642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d05ad-b275-4967-a1c4-d4bae2c21761 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.618983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 94/177] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:19.628201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.628548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.628816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.664215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:19.666470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:19.667592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:19.679037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 84/178] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:19.679983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.806026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362e28-f685-474b-b9fd-5447895f478b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.816081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 95/179] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:19.817703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.848822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.850151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.850652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.851146 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/eea1a45cc08d4b8da5906ae469514256/roles/8953d2c24d7c4f8696f1cbfeeae409f0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.855439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, user_id=eea1a45cc08d4b8da5906ae469514256, role_id=8953d2c24d7c4f8696f1cbfeeae409f0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.857257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.874764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 85/180] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:19 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/users/eea1a45cc08d4b8da5906ae469514256/roles/8953d2c24d7c4f8696f1cbfeeae409f0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:19.876049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.957383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.001455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.037141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392dcac-1ef3-4599-8161-45c55936d324 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.045127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39746d4e-2655-4ec5-b167-07f66dd0c619 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.089249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.090500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.090953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.091372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.102497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.103903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.104417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.104719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:20.104898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] PATH_INFO: `/v3/projects/1fe219c085b743eaae0506e97f8ba868` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.107124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] RBAC: Authorizing `identity:get_project(project_id=1fe219c085b743eaae0506e97f8ba868)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:20.109944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a356050-5a2b-42ef-8bb0-6a45a6663c49 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-903438cb-68b7-471b-9487-97c2ed66b303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:20.112703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:20.113734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:20.114543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:20.115160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 96/181] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/projects/1fe219c085b743eaae0506e97f8ba868 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:20.115231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 86/182] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:20.120058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5fba-ca97-48e6-86f0-8a17eeeb6968 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.120230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d520e3-d248-48d9-a4fb-23e3ededd4b2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.120308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5fba-ca97-48e6-86f0-8a17eeeb6968 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.120308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5fba-ca97-48e6-86f0-8a17eeeb6968 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.120308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 87/183] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:20.124196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.124196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.124196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.126263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d520e3-d248-48d9-a4fb-23e3ededd4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.126482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d520e3-d248-48d9-a4fb-23e3ededd4b2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.129342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 97/184] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:20 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:20.133131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d520e3-d248-48d9-a4fb-23e3ededd4b2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.147976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:20.149591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:20.272994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 88/185] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:20.286834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.312021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d520e3-d248-48d9-a4fb-23e3ededd4b2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.354579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259aae-b398-49dd-a961-570327024744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.355846 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.356733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259aae-b398-49dd-a961-570327024744 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.357360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259aae-b398-49dd-a961-570327024744 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.357808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259aae-b398-49dd-a961-570327024744 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.358627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259aae-b398-49dd-a961-570327024744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:20.358985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259aae-b398-49dd-a961-570327024744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:20.360975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259aae-b398-49dd-a961-570327024744 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:20.362734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.381714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 98/186] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:20.381714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.381714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.381714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.406775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:20.408943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:20.440115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a8c97-156c-445a-b24d-3d73722c53b4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.500674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.501885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.502295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.502700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.504301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=1fe219c085b743eaae0506e97f8ba868, user.id=d41522a9f214417fad9710b6bfce9f18) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:20.504747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:20.506660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:20.521724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 89/187] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/role_assignments?user.id=d41522a9f214417fad9710b6bfce9f18&scope.project.id=1fe219c085b743eaae0506e97f8ba868&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:20.521724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.529362 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 99/188] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:20.531249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.729978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10646aa-d96e-44c6-92bd-3dfd4f7c20d5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.751498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1181-444e-4baf-94c5-6e32c956ca10 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.775871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.787141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.808831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.808831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.808831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.809604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.825894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.827176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.827691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.828262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.842244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:20.844293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:20.847379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 100/189] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:20.848807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.858070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:20.874409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:20.876140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 90/190] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:20.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.044076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d52cc-b5b4-4ead-8f24-2dc2ed600a8d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.057654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dc86-ca0b-416d-a92f-3cd326a16518 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.105525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.106661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.107160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.107689 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.110857 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.112059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.112476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.114050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.115572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.117856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.120802 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 101/191] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:21.122170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.174169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.196794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.216860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.218763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.260906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fed58-e55e-4c36-a177-f1e27155c018 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.315721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.317165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.317613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.318037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.319249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:21.319776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.322182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.336652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 102/192] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:21.338060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.397027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 91/193] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:21.398592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.473324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd55ee-02b1-41ea-8078-6e7089da9da9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.496342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.522330 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.523019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.523309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.523588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.524318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.525811 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.527254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.555366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 103/194] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:21 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:21.557299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.588640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1e55-84f0-40f1-a7c8-b6f52ab11a6a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.657647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.658848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.659345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.660457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.662158 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.663909 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.665957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 92/195] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:21.676584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.738775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b429a-9581-45c2-95dd-ba80de69b93f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.787377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.788980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.791312 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.791312 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.793276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.797706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.806297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 104/196] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:21.807694 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.880441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.920903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4a881-8564-4a8a-9acc-b53d2f3ea4ee None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.992356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b665a-d091-43c8-bc61-9afb4a1046ee None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.997453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.997928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.998796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.998796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.019554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:22.052582 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:22.053799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.054260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.054715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.057206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=eaf2d82bca614418a8155729b1f4f06e) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:22.057866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:22.061168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:22.077103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 105/197] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/role_assignments?user.id=eaf2d82bca614418a8155729b1f4f06e&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:22.157316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:22.160374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:22.377482 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 93/198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:22.380241 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:22.674011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dcb59-4d1e-47b9-b0ca-6a479041dea7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:22.677541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:22.762600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:22.765734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.766643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.767376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.775283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:22.778383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4f04-6971-469e-8b4b-af968576da61 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:22.781948 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 106/199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:22 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:22.791799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:22.817404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abfa0-734f-4adf-a920-d2d2e51640a1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.818306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abfa0-734f-4adf-a920-d2d2e51640a1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.819027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abfa0-734f-4adf-a920-d2d2e51640a1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.822040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 107/200] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:22.830832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.832324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.833120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.880861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:22.882875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:22.923035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:23.124355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fc0c-2ffd-42f6-ba23-698efc91e037 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.151351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 108/201] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:23.202090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.203451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.203934 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.204411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.206038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=eaf2d82bca614418a8155729b1f4f06e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:23.207172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.209967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.221012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 94/202] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:22 2022] GET /identity/v3/role_assignments?user.id=eaf2d82bca614418a8155729b1f4f06e&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 259 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:23.231535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.497363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:23.502160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aca310-a129-47fa-98e4-9b90d9ad6f8d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.575514 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.577396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.578234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.579158 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.608037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:23.628292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:23.630445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:23.631946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:23.640629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 109/203] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:23.650894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.910765 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48123a2-4b81-47c7-bd4f-921577c88f0d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.913574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:23.997814 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.998906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.999297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.999807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.001257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:24.001864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.004684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.028312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 95/204] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:24.035925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.251618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5f65-ff1b-4adf-98a5-b10e130e4ca4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.323657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.325254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.325749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.326232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] PATH_INFO: `/v3/users/eea1a45cc08d4b8da5906ae469514256` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.331089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] RBAC: Authorizing `identity:get_user(user_id=eea1a45cc08d4b8da5906ae469514256)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.334703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.337834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 110/205] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:24 2022] GET /identity/v3/users/eea1a45cc08d4b8da5906ae469514256 => generated 289 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:24.343595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.579505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30b15-c11f-41e6-822c-ddbeaafb3f15 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.618068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.619126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.619624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.620119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.656617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:24.702548 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.704510 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838634c-6ba6-425d-b129-9dcaf540459b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.831783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 96/206] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:53:24.839480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.910414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:25.072604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b45e39-9d12-4432-880d-53d313d4933e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:25.087872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eb7c4-e49a-4db1-b733-e133bdfb9259 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.088832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eb7c4-e49a-4db1-b733-e133bdfb9259 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.089361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eb7c4-e49a-4db1-b733-e133bdfb9259 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.091378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 97/207] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:25.097910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48d17-0cf1-491e-88ec-e4bec0e749cd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.098516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48d17-0cf1-491e-88ec-e4bec0e749cd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.098993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48d17-0cf1-491e-88ec-e4bec0e749cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.148342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:25.149926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.150481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.150985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.153615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:25.156443 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:25.164046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 111/208] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:24 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:25.169314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-55c48d17-0cf1-491e-88ec-e4bec0e749cd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:25.171347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55c48d17-0cf1-491e-88ec-e4bec0e749cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:25.176387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:25.356078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:25.369408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:25.383160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde9cd-20ff-4399-97cf-8b13ade9d920 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:25.386303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 98/209] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:25.430014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed4fd-c06d-4dcb-9e1e-cb13d1cb52e8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.430687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed4fd-c06d-4dcb-9e1e-cb13d1cb52e8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.431179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed4fd-c06d-4dcb-9e1e-cb13d1cb52e8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.432851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 99/210] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:25.433938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed4fd-c06d-4dcb-9e1e-cb13d1cb52e8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:25.454125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:25.456054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.457133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.457849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.460055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63, user.id=eea1a45cc08d4b8da5906ae469514256) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:25.460898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:25.464365 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8944b-acbb-42e4-9f35-c0f59123d163 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:25.480565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 112/211] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:25 2022] GET /identity/v3/role_assignments?user.id=eea1a45cc08d4b8da5906ae469514256&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:25.485822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.485822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.485822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.518016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:25.521134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:25.649839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed4fd-c06d-4dcb-9e1e-cb13d1cb52e8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:25.723645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:25.725179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.726145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.726887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:25.739200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 113/212] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:25.748800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:25.768426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:25.770854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:25.772560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:25.775186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 100/213] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:25.776894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:25.781409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:25.875354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:25.944342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:25.958522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9aff3d-4686-4a2d-8ca5-d4a403e34d3d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:25.974814 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef774f-e0f6-4c9c-a03f-86fc48574b14 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:25.998483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:25.999317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:25.999601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:25.999882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.013798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:26.023339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:26.024296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:26.025088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:26.027725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 114/214] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:26.033720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.051759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.053079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.053724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.054357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.055907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:26.056577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.059263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.076960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 101/215] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:26.082782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.173481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65889d-9564-4069-86d8-9b9ad6cb795f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.224184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.225138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.225138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.225429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.226442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:26.226733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.228483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.243039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 115/216] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:26.244180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.254702 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11934-42a8-4f34-ad05-79ab20d7d634 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.263342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.282173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.304015 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.304905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.305194 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.305474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] PATH_INFO: `/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.320663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] RBAC: Authorizing `identity:get_user(user_id=2b4b9fa9a578466599dc1e98a9a52bfe)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.322285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cb232-5eca-4d3e-9306-2c6add536972 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.324154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 102/217] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:26.330062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3fe5a-f85c-4a22-b86c-6508cc1a58dc None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.330358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3fe5a-f85c-4a22-b86c-6508cc1a58dc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.330629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3fe5a-f85c-4a22-b86c-6508cc1a58dc None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.332685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 103/218] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:26.333426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3fe5a-f85c-4a22-b86c-6508cc1a58dc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.363333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce061e-b272-4b40-b57e-ebd215c892dd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.401686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.402706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.402985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.403263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.404634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.406321 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48768-a753-43b6-86b7-1c7bc6e4dbce None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.412650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 116/219] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:26.412650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.412650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.412650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.428367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:26.429571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:26.453030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3fe5a-f85c-4a22-b86c-6508cc1a58dc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.492030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.492811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.493102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.493389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.501929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 117/220] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:26.514797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.536661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.544018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.582633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.584703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.650603 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b48a5c-e9ea-4ef8-a95a-3d6c04e3f996 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.692145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.693025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.693314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.693591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.694832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.696300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.698256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 118/221] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:26.698890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.706766 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 104/222] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:26.707700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.790651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.794974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:26.821474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab47d3-1591-4308-ac4e-792ae3979354 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.823135 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb6ccb-7676-4b5e-86a8-8a70a171ee96 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:26.861928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.862269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:26.863881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.864106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.864340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.864572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.864793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/eaf2d82bca614418a8155729b1f4f06e/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.865022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.866970 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.867992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, user_id=eaf2d82bca614418a8155729b1f4f06e, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:26.869889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.870405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99e46f-9b06-492e-b21a-bc73875f3c2e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:26.872843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 119/223] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:26.874217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:26.888400 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 105/224] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:26 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/eaf2d82bca614418a8155729b1f4f06e/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:26.889828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0e1-c4ff-45ef-8711-8dcc53d4f9f9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:26.890117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0e1-c4ff-45ef-8711-8dcc53d4f9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:26.890395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0e1-c4ff-45ef-8711-8dcc53d4f9f9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:26.891713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 106/225] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:26.892459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b0e1-c4ff-45ef-8711-8dcc53d4f9f9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:27.011728 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6d263-2faa-4b86-a94c-9b573add44a3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:27.042872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b0e1-c4ff-45ef-8711-8dcc53d4f9f9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:27.068902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:27.069956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.070244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.070522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.072109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:27.073457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195def81-d7e6-466a-aa04-519c64c40ef6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:27.075823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 120/226] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:26 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:27.076805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.077095 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.077378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.091061 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:27.092547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.092547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.092547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.092999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:27.095193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:27.103518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:27.104297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:27.107781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:27.124418 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 107/227] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:26 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:27.129729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:27.133610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:27.208869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 121/228] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:27.252910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:27.268513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc8538-24ec-4d89-8078-a63cc8f88ed0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:27.340328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:27.341118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.341432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.341790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.342714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=2b4b9fa9a578466599dc1e98a9a52bfe) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:27.343036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:27.345021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:27.353600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 108/229] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:27 2022] GET /identity/v3/role_assignments?user.id=2b4b9fa9a578466599dc1e98a9a52bfe&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:27.439515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ece8c2-fdb2-4117-b722-9f6082e2feaf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:27.506035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:27.507978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.508800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.509490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.531379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:27.548758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:27.550818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:27.552442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:27.559725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 122/230] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:27.569362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:27.642578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:27.703352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb1882-e808-4084-b026-893ddc7ac668 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:27.746499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:27.747196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:27.747483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:27.747770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:27.748542 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:27.748860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:27.750352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:27.761511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 109/231] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:27.771014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:27.778826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:27.932567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b02-a14c-416d-b220-84395f35d2d1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:28.006911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:28.009034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:28.009873 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:28.010657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:28.013292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:28.016517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:28.021915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 123/232] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:27 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:28.029252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:28.197713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d2957-2e5b-495a-bb98-bce21807ac19 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:28.220859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:28.250632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:28.251526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:28.251818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:28.252101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:28.257193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:28.258735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6a73c-43c6-4516-b9b3-e948251f2b5c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:28.260961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 110/233] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:28 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:28.270021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:28.373024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:28.419532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6d208-0347-40e0-8b68-54c9c0ac7f2d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:28.471079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:28.472465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:28.472759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:28.477983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/eaf2d82bca614418a8155729b1f4f06e/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:28.480630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, user_id=eaf2d82bca614418a8155729b1f4f06e, role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:28.482150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6e5cb-32c3-4838-b71e-04ec417e651a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:28.506839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 124/234] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:28 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/eaf2d82bca614418a8155729b1f4f06e/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:29.849050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72026-73a5-4be4-ab53-de00999b915b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:29.849050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72026-73a5-4be4-ab53-de00999b915b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:29.849050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72026-73a5-4be4-ab53-de00999b915b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:29.850279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 111/235] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:29.852043 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d165547-61e9-48d6-a572-477848d207b6 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:29.852832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d165547-61e9-48d6-a572-477848d207b6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:29.853305 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d165547-61e9-48d6-a572-477848d207b6 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:29.854984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 125/236] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:29.860594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:29.861191 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:29.861687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:29.867509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:29.876472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:29.877100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:29.889287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:29.891831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:29.911286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:29.913101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:29.994345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 112/237] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:30.016807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 126/238] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:30.040585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.052741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.175682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:30.205777 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f8bdf-51c6-4204-8309-1ff855708114 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.223597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:30.243430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f9c73-36d7-4563-99a5-f7eecc161f13 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.248984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.250126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.250605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.251043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.263957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-49e6a8cc-2260-4775-898c-3c05a40f2831 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:30.272684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:30.273804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:30.274625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:30.277435 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 113/239] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:30.279090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0b02-8098-447d-a501-4ff270264a7d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.279389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0b02-8098-447d-a501-4ff270264a7d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.279672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0b02-8098-447d-a501-4ff270264a7d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.285711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 114/240] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:30.287293 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c0b02-8098-447d-a501-4ff270264a7d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.292459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.292459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.292837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.293044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.306972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c4cb39ff-4bfc-4507-8b57-c8d0e6027f53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:30.315216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:30.318181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 127/241] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:30.318996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.320799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.320799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.337381 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:30.338710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:30.418536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 128/242] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:30.419329 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.428690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c0b02-8098-447d-a501-4ff270264a7d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.480282 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.481106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.481436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.481746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.482525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:30.482838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:30.484497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9b68-9eba-48d3-9d3e-acf3fbf69d52 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:30.499692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 115/243] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:30.499692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726804-56ef-4175-b141-a168e7c047d9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.499692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726804-56ef-4175-b141-a168e7c047d9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.499692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726804-56ef-4175-b141-a168e7c047d9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.499692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 116/244] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:30.500338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726804-56ef-4175-b141-a168e7c047d9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.501919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:30.547849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:30.567736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcecdd-2aa0-43bb-9b89-8f87f042a4fd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.615266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.616040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.616388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.616729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.617573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:30.617996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:30.620255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:30.637857 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 129/245] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:30.637857 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.639328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726804-56ef-4175-b141-a168e7c047d9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.705516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.708011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.708851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.709987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.712522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:30.715401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192bcc2-351a-48bc-a13a-95488f222bfd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:30.726463 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 117/246] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:30.733311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.733311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.733311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.759374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:30.761663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:30.805181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380d12-a16b-4669-bdc4-3e045b756e58 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:30.840488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:30.874213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 118/247] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:30.875694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.889211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:30.891751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:30.893202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:30.894597 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] PATH_INFO: `/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:30.897650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] RBAC: Authorizing `identity:get_user(user_id=2b4b9fa9a578466599dc1e98a9a52bfe)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:30.900570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:30.904395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 130/248] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:30.906129 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:30.939249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:31.064121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59b0da-92c1-403e-bb6f-7951ed05d4d6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.077706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01705646-744e-4904-b698-df96a2a98e15 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.137609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.138677 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.139101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.139540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.140496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:31.141403 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.142363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:31.143044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.143578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.144160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.146678 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:31.149771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:31.155227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 119/249] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:30 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:31.158822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:31.180106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 131/250] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:30 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:31.182557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:31.248580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:31.332710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d99b-d3c3-49a0-9be2-53cb6eeb01bf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.336476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:31.353270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438773f3-58a0-460b-8182-a0158f2f1eda None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.403442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.404636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.405113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.405587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.407636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:31.410044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.410749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:31.411452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.411974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.412485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.413030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 120/251] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:31 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:31.414291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:31.428212 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-74b14dcf-667a-43b6-af35-e7feb0b1563b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:31.442619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:31.443562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:31.444348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:31.444946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 132/252] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:31.446765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af920c-1acc-40fb-a811-eb4d173d3859 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.447281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af920c-1acc-40fb-a811-eb4d173d3859 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.447775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af920c-1acc-40fb-a811-eb4d173d3859 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.450256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 133/253] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:31.451478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af920c-1acc-40fb-a811-eb4d173d3859 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:31.583569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c27c0-6554-44f5-8926-deea08c94f56 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.607656 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:31.621261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af920c-1acc-40fb-a811-eb4d173d3859 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:31.651297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.652497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.652970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.653779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.690764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:31.692440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.693381 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.693895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/2b4b9fa9a578466599dc1e98a9a52bfe/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.697306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, user_id=2b4b9fa9a578466599dc1e98a9a52bfe, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:31.700658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695397d-40cc-4632-b159-039c5a5add01 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:31.727830 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 134/254] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:31 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/2b4b9fa9a578466599dc1e98a9a52bfe/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:31.735349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:31.737454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:31.738587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:31.741851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:31.798866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:31.800670 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:31.811348 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:31.814447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:31.995915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 135/255] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:31.997832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.077457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 121/256] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:32.078401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.232012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:32.242744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:32.254994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af323b8-84ef-4cb6-8b01-2ea9e6b9bf58 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:32.317655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:32.318704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b253eb-2d1d-4a2e-a968-d853e82c3b6c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:32.319528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:32.320293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:32.321024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:32.322777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:32.323657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:32.326633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:32.345548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 136/257] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:32.351383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.397853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:32.399090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:32.399525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:32.399866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:32.417589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:32.431521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:32.432744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:32.434131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:32.436815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 122/258] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:32.438590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.569983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a88e6a-e7d4-450b-84b1-893174160fe7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:32.651691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:32.653234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:32.653768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:32.654261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:32.660039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:32.662794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:32.674761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 137/259] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:32 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:32.679030 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.718195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2df8a-6631-4b7c-a708-9ea264413812 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:32.756608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:32.779704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:32.802949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:32.804503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:32.805012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:32.805527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] PATH_INFO: `/v3/users/1ffbd366fd9341c78967dccdd65fbccd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:32.865840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=1ffbd366fd9341c78967dccdd65fbccd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:32.867421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:32.869344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 123/260] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:32 2022] GET /identity/v3/users/1ffbd366fd9341c78967dccdd65fbccd => generated 297 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:32.870295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:32.898905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27a39-9751-4405-9599-5fe5f8509e92 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:33.001724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:33.002759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:33.002759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:33.002759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:33.003177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:33.003501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:33.004944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:33.014883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 138/261] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:33.015778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:33.085778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfd7b3-338c-4cb5-b13e-fb1f86b95cc3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:33.114979 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:33.146472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64214ecc-de44-4447-bd61-43007f6119af None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:33.185346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:33.186510 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:33.187009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:33.187503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:33.189172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=eaf2d82bca614418a8155729b1f4f06e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:33.189777 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:33.192573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:33.220685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 124/262] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:32 2022] GET /identity/v3/role_assignments?user.id=eaf2d82bca614418a8155729b1f4f06e&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:33.225231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:33.255226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:33.255923 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:33.256622 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:33.256919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:33.344420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:33.442363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f95d-462d-4ed0-af5d-67111a7b44ce None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:33.463350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:33.466048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:33.518334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:33.520357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:33.520870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:33.521362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] PATH_INFO: `/v3/users/eaf2d82bca614418a8155729b1f4f06e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:33.524880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] RBAC: Authorizing `identity:get_user(user_id=eaf2d82bca614418a8155729b1f4f06e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:33.527759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:33.531012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 125/263] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:33 2022] GET /identity/v3/users/eaf2d82bca614418a8155729b1f4f06e => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:33.538448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:33.618019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:33.645768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 139/264] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:33.654922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:33.783380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ce6e-e632-4769-8ecd-56b4e734f274 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:33.827937 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:33.829300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:33.829617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:33.829911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:33.831947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:33.833566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:33.836660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 140/265] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:33 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:33.850487 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:33.913406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83f0b-766f-4ec5-a89f-1d68d2c512bd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:34.004763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:34.033835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:34.056386 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:34.058023 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:34.061706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:34.062557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:34.222367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30eb3d-fed2-4cdc-ab97-15656dcab79c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:34.315836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:34.320214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:34.357179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:34.358297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:34.358798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:34.359311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:34.360972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=1ffbd366fd9341c78967dccdd65fbccd) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:34.361520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:34.364308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec447e-687d-43d8-a5ec-76429ecda234 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:34.375071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 141/266] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:33 2022] GET /identity/v3/role_assignments?user.id=1ffbd366fd9341c78967dccdd65fbccd&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 259 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:34.377490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066928be-0b31-42fe-b1ee-527b03e3acde None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:34.378050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066928be-0b31-42fe-b1ee-527b03e3acde None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:34.378593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066928be-0b31-42fe-b1ee-527b03e3acde None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:34.381459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 142/267] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:34.391116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:34.391621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:34.392134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:34.436906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:34.439102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:34.607987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 126/268] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1070 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:34.619735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:34.628774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 143/269] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:34.671562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:34.723284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:34.834173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf745-2a60-4dfb-90e4-547bf700f8ef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:34.848768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:34.912072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:34.913533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:34.914082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:34.914591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:34.918448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:34.919961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:34.921820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 127/270] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:34 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:34.928365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:35.000571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e74c44-3852-467f-be4f-d904fd8a13c4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.068962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:35.070354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.070863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.071373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.091629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bd678bb1-34ce-4ed8-9592-df2bcef3700d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:35.107372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:35.109240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:35.111336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:35.114577 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 144/271] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:35.131336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f146b-08d5-45b8-b23b-76f4cf155e70 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.131836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f146b-08d5-45b8-b23b-76f4cf155e70 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.132328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f146b-08d5-45b8-b23b-76f4cf155e70 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.135189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 145/272] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:35.140658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f146b-08d5-45b8-b23b-76f4cf155e70 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:35.205211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:35.231282 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693b97a-25f9-420f-8c0d-a1b9e53f89e5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.304373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503fd19-6179-402f-b48f-38895471115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:35.305992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fd19-6179-402f-b48f-38895471115f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.306796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fd19-6179-402f-b48f-38895471115f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.307694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fd19-6179-402f-b48f-38895471115f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.309764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fd19-6179-402f-b48f-38895471115f None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=eaf2d82bca614418a8155729b1f4f06e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:35.310706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fd19-6179-402f-b48f-38895471115f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:35.313902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fd19-6179-402f-b48f-38895471115f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:35.326301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 128/273] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:34 2022] GET /identity/v3/role_assignments?user.id=eaf2d82bca614418a8155729b1f4f06e&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:35.329701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d002b-0dad-4769-b693-abfe9605e359 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.330527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d002b-0dad-4769-b693-abfe9605e359 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.331472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d002b-0dad-4769-b693-abfe9605e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.347169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f146b-08d5-45b8-b23b-76f4cf155e70 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.350613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:35.370020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-585d002b-0dad-4769-b693-abfe9605e359 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:35.372802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-585d002b-0dad-4769-b693-abfe9605e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:35.418008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:35.419127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.419625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.420772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.421716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:35.422265 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:35.425565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:35.464347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 146/274] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:35.474429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:35.579117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 129/275] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:35.586628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e4d5d-b9a8-49e7-aef9-4ab8f185168e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.587489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e4d5d-b9a8-49e7-aef9-4ab8f185168e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.588259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e4d5d-b9a8-49e7-aef9-4ab8f185168e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.591537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 130/276] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:35.599407 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.600351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.601149 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.629861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:35.631255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:35.673703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c1cfd-c4c7-4b7e-9110-0ce5dcb815f5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.733104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:35.734014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.734266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.734551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] PATH_INFO: `/v3/users/80a06e2e516e41ecbed6078f06d9bc25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.746102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 131/277] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:35.746102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:35.750801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] RBAC: Authorizing `identity:get_user(user_id=80a06e2e516e41ecbed6078f06d9bc25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:35.752729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:35.754689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 147/278] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:35 2022] GET /identity/v3/users/80a06e2e516e41ecbed6078f06d9bc25 => generated 293 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:35.766737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:35.777828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:35.811692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:35.893939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4ea9e-cef2-44f3-9de8-e7e9469f966d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.951960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:35.952778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:35.953068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:35.953350 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:35.954714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:35.955927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101035-e623-43cd-b28d-d8148b9ebac7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:35.957060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:35.959736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 132/279] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:35.960632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:36.025896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:36.027035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:36.027533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:36.028022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:36.029615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, user.id=80a06e2e516e41ecbed6078f06d9bc25, scope.system=all) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:36.030227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:36.033289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:36.042079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 148/280] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 11 11:53:35 2022] GET /identity/v3/role_assignments?user.id=80a06e2e516e41ecbed6078f06d9bc25&scope.system=all&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:36.043230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:36.101620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:36.170950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea89e7-a2dd-4584-99c6-9da5043c9fdf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:36.254404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:36.256296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:36.257725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:36.260992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:36.261843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:36.291720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00e578-7715-45de-a453-e8c9376f08fc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:36.291892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:36.308204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:36.310358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:36.311351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 133/281] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:36.321448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:36.356700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:36.357915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:36.358420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:36.358900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:36.359872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:36.361892 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:36.394151 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 149/282] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:53:36 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:36.553878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87dabf-b716-4cfc-be2e-1a00a80ff3e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:36.630894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:36.632069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:36.632560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:36.633042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:36.634589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:36.635142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:36.637832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:36.659885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 134/283] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:36.668394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:36.824488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cab6-1a00-40be-9bbf-0d61393dd5e4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:36.885391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:36.887230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:36.887968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:36.888683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] PATH_INFO: `/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:36.891392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] RBAC: Authorizing `identity:get_user(user_id=2b4b9fa9a578466599dc1e98a9a52bfe)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:36.894460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e53de1-1b85-4951-8f1c-b1003e704f14 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:36.896594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 150/284] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:36 2022] GET /identity/v3/users/2b4b9fa9a578466599dc1e98a9a52bfe => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:36.906117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:36.911907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:37.055838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276668c-4048-4930-83a5-69caeb9a4b04 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:37.056652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276668c-4048-4930-83a5-69caeb9a4b04 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:37.057357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276668c-4048-4930-83a5-69caeb9a4b04 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:37.061302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 151/285] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:37.070468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ca6d-74c9-4146-9866-929f7edbbeea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:37.084680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:37.085090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:37.085598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:37.141446 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:37.146684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:37.154097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:37.156401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:37.157100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:37.157387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:37.157713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:37.304974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 152/286] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:37.341236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:37.353559 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:37.355108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:37.358450 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:37.531452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 135/287] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:37.537247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:37.580703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e8782-8a1a-4289-bb86-a9d9caf2fab5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:37.647965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:37.649804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:37.650525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:37.651309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:37.673045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:37.683674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e5a0-548e-4bdf-a63a-9b009b2035fa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:37.685426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:37.686049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:37.713027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:37.714059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:37.714850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:37.715963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 153/288] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:37.723633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:37.734803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc23075-125e-4736-a18e-774414908607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:37.735719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23075-125e-4736-a18e-774414908607 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:37.736009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23075-125e-4736-a18e-774414908607 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:37.736285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23075-125e-4736-a18e-774414908607 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:37.744528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc23075-125e-4736-a18e-774414908607 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:37.748346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc23075-125e-4736-a18e-774414908607 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:37.757726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 136/289] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:37 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:37.763377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc23075-125e-4736-a18e-774414908607 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:37.968040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14191212-3519-4a2e-8556-4991440b1284 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:37.994657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc23075-125e-4736-a18e-774414908607 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:38.041475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:38.042207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:38.042499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:38.042776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:38.043630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:38.043876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:38.045315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:38.062535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:38.063225 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:38.063513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:38.063793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:38.064651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=2b4b9fa9a578466599dc1e98a9a52bfe) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:38.064970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:38.068430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 154/290] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:38.068845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:38.078107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:38.078601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 137/291] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:37 2022] GET /identity/v3/role_assignments?user.id=2b4b9fa9a578466599dc1e98a9a52bfe&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:38.191567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:38.256776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0afac-a285-4636-b3a4-a0629a541714 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:38.315252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:38.316754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:38.317141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:38.317318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] PATH_INFO: `/v3/users/1ffbd366fd9341c78967dccdd65fbccd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:38.322838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] RBAC: Authorizing `identity:get_user(user_id=1ffbd366fd9341c78967dccdd65fbccd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:38.324354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:38.326244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 155/292] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:38 2022] GET /identity/v3/users/1ffbd366fd9341c78967dccdd65fbccd => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:38.336218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:38.391297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:38.474845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623843ef-e0ab-432d-b03f-b0e4a87031d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:38.514658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:38.515440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:38.515728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:38.516008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:38.517271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:38.519371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3729c13-cccc-41df-88ef-2aa16d70193a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:38.521183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 138/293] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:38 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:38.527089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:38.651345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993140e-5803-437b-9977-1a1806178bb1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:38.683972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:38.696098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:38.697024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:38.697311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:38.697591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/1ffbd366fd9341c78967dccdd65fbccd/roles/8953d2c24d7c4f8696f1cbfeeae409f0` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:38.700977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, user_id=1ffbd366fd9341c78967dccdd65fbccd, role_id=8953d2c24d7c4f8696f1cbfeeae409f0)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:38.702480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f7b9a-1b39-4456-a6bc-42bd4ad89ab0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:38.730079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 156/294] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:53:38 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/users/1ffbd366fd9341c78967dccdd65fbccd/roles/8953d2c24d7c4f8696f1cbfeeae409f0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:53:39.434264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836008-558c-411f-a57d-bedb84f26c44 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.434779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836008-558c-411f-a57d-bedb84f26c44 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.434779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836008-558c-411f-a57d-bedb84f26c44 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.436215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 139/295] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:39.442290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c1a4-c978-4d1a-9f2c-39af0a90d751 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.442823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c1a4-c978-4d1a-9f2c-39af0a90d751 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.442919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c1a4-c978-4d1a-9f2c-39af0a90d751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.461854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b2a6c1a4-c978-4d1a-9f2c-39af0a90d751 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:39.463093 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b2a6c1a4-c978-4d1a-9f2c-39af0a90d751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:39.549833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 157/296] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:39.565983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836008-558c-411f-a57d-bedb84f26c44 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:39.753305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836008-558c-411f-a57d-bedb84f26c44 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:39.755877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:39.773358 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a036-ec8b-4a99-8f62-5f5e0ad2a809 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.774125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a036-ec8b-4a99-8f62-5f5e0ad2a809 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.774625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a036-ec8b-4a99-8f62-5f5e0ad2a809 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.776671 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 158/297] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:39.782557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a432b02-bad9-47d2-9f23-6444d2ac596d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.783269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a432b02-bad9-47d2-9f23-6444d2ac596d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.783783 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a432b02-bad9-47d2-9f23-6444d2ac596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.802444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4a432b02-bad9-47d2-9f23-6444d2ac596d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:39.804010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4a432b02-bad9-47d2-9f23-6444d2ac596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:39.816237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:39.817343 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.817839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.818268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.837928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:39.853252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:39.854331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:39.855174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:39.856177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 140/298] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:39.867415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:39.889487 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 159/299] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:39.899838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926dc4-8590-46b4-8fb4-fe662c92d983 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.900428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926dc4-8590-46b4-8fb4-fe662c92d983 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.900919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926dc4-8590-46b4-8fb4-fe662c92d983 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.904308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 160/300] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:39.911384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:39.911691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:39.911963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:39.935465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:39.937804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:40.002088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:40.029448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 161/301] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:40.029918 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acdaea-fe2c-4945-ac21-6d1dfdea0671 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.034978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.078505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.079213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.079485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.079771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.080546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:40.080887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.082900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.097531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 141/302] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:40.097531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.138204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:40.193398 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d88ba7-0c7e-424e-bbfe-eada21c0222b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.232391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.233202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.233497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.233824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.246395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:40.255206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:40.256204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:40.257006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 162/303] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:40.257754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.276325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffd96e-3e2a-46c2-9740-3510ace5326c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.303327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:40.337729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.339101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.339601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.340097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.342287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.345056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.348282 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 142/304] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:40.348282 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.378417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b60b0-01f7-421d-9f01-8c55958f2d72 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.415691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.416593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.416883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.417164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] PATH_INFO: `/v3/users/80a06e2e516e41ecbed6078f06d9bc25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.418469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] RBAC: Authorizing `identity:get_user(user_id=80a06e2e516e41ecbed6078f06d9bc25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.419971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.427052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 163/305] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/users/80a06e2e516e41ecbed6078f06d9bc25 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:40.427052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.508586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:40.523713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62a24-f245-4e1f-a788-a5d8f26933b8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.553473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a316d-dacc-4e2e-8ca7-4a145d32aa57 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.563927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:40.580038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.580722 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.581052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.581338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.582413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:40.582546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.583892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.593506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 143/306] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:40.594320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.611768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.613639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.614184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.614662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] PATH_INFO: `/v3/groups` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.616175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.619243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.650606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 164/307] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 11 11:53:40 2022] POST /identity/v3/groups => generated 229 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:40.650606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.757744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3f1f-e655-45b2-84ea-070b94e23010 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.805499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.806437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.806729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.807011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] PATH_INFO: `/v3/system/users/80a06e2e516e41ecbed6078f06d9bc25/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.808858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=80a06e2e516e41ecbed6078f06d9bc25, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.810383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.828255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 144/308] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 11 11:53:40 2022] PUT /identity/v3/system/users/80a06e2e516e41ecbed6078f06d9bc25/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:40.863082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a078b0-4309-4f05-a5b6-1340e61254fa None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:40.916097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:40.917085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:40.917372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:40.917656 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] PATH_INFO: `/v3/users/9024cc5d68eb40318890a244a68dc683` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:40.934746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] RBAC: Authorizing `identity:get_user(user_id=9024cc5d68eb40318890a244a68dc683)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:40.936392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afe793-05d9-4386-9494-2c617c7a7d13 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:40.938771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 165/309] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/users/9024cc5d68eb40318890a244a68dc683 => generated 293 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:40.945049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:40.972142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:41.077746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7296123-4e8f-4372-92ca-10b2ee6f372a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:41.120202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:41.120894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:41.121186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:41.121471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:41.122417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, user.id=9024cc5d68eb40318890a244a68dc683, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:41.122745 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:41.124296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061f31f-85ea-43f4-a3f1-8776031ca25b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:41.135313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 145/310] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 11 11:53:40 2022] GET /identity/v3/role_assignments?user.id=9024cc5d68eb40318890a244a68dc683&scope.system=all&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:41.688826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e670bd-9826-4902-9c75-4087aa8a4ed3 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:41.689396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e670bd-9826-4902-9c75-4087aa8a4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:41.689835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e670bd-9826-4902-9c75-4087aa8a4ed3 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:41.693005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 166/311] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:41.701717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:41.702237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:41.702716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:41.734518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:41.734518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:41.877048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 146/312] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:41.914191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e670bd-9826-4902-9c75-4087aa8a4ed3 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:41.952655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:42.044911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e670bd-9826-4902-9c75-4087aa8a4ed3 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:42.086785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:42.087871 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:42.088339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:42.088785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:42.102210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:42.110736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:42.111928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:42.113501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:42.116011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 167/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:42.125844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:42.151995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:42.303716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3310-ce4c-476f-aca9-e4b75ba7aa56 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:42.369990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:42.371105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:42.371615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:42.372113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:42.373586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:42.374184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:42.376863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:42.394411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 147/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:42.403034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:42.511094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:42.532106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30e4a-da70-4ee5-8653-0131fca31a70 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:42.570075 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:42.571074 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:42.571365 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:42.571644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] PATH_INFO: `/v3/users/1ffbd366fd9341c78967dccdd65fbccd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:42.572927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=1ffbd366fd9341c78967dccdd65fbccd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:42.574442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:42.576461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 168/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:42 2022] GET /identity/v3/users/1ffbd366fd9341c78967dccdd65fbccd => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:42.586013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:42.691176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:42.730755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e20787-00f8-4afc-8dd3-43087146170e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:42.787236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:42.787911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:42.788209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:42.788483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:42.869438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:42.870907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a96f-f49a-4ab0-821f-c6504791636e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:42.967380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:43.000749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 148/316] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:43.007701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:43.126251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648066a-21d3-4e79-88d9-5df662695832 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.126712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648066a-21d3-4e79-88d9-5df662695832 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.127100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648066a-21d3-4e79-88d9-5df662695832 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.134096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 149/317] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:43.139121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.139554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.140159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.161697 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:43.163581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:43.179408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0fc3-ed8e-4644-8b80-db478eb50ad2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:43.235162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:43.236625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.237554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.237979 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.239650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:43.241362 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:43.244090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 169/318] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:43 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:43.250326 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:43.278091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:43.281320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 150/319] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:43.305100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:43.389471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722f36-0f0e-4842-b9c7-a239db91f408 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:43.457091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:43.458290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.458794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.459293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.460639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, scope.project.id=2c65192a7afb43748595114bfe9ea9bd, user.id=1ffbd366fd9341c78967dccdd65fbccd) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:43.461172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:43.462924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761094b8-825a-4da8-9002-6d388b2ee178 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:43.477826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 170/320] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:53:43 2022] GET /identity/v3/role_assignments?user.id=1ffbd366fd9341c78967dccdd65fbccd&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:43.503976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a4ba9-9512-43cb-8a65-9206a0ad4747 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:43.505876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0d4d-0168-4255-8079-1fabdb3aa4bd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.506382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0d4d-0168-4255-8079-1fabdb3aa4bd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.506896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0d4d-0168-4255-8079-1fabdb3aa4bd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.508458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:43.513266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 171/321] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:43.515699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.515998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.516309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.558769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:43.561087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:43.563311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:43.564425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.565014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.565536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.567180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:43.568972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:43.571531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 151/322] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:53:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:43.581806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:43.710203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3ddd8-167a-4907-95c8-85a5edc648fa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:43.723225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 172/323] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:43.749331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:43.758204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:43.759389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.759911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.760433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] PATH_INFO: `/v3/groups` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.761870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:43.764695 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:43.776806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:43.797376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 152/324] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 11 11:53:43 2022] POST /identity/v3/groups => generated 226 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:43.911645 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf05dae-42a1-4e79-a52e-822885b3f508 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:43.956633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:43.957462 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:43.957788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:43.958065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:43.972353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:43.980488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:43.982310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:43.983946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 173/325] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:43.988908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:44.035457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:44.125319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41121d6-9021-4d22-8977-8de81c0cf08a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:44.192627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:44.193840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:44.194353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:44.194845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:44.196289 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:44.196860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:44.199642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:44.216999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 153/326] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:44.224610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:44.395338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:44.447468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a72519-c3ef-4e5a-b3fb-62294598e4b1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:44.522119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:44.523036 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:44.523377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:44.523668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] PATH_INFO: `/v3/users/80a06e2e516e41ecbed6078f06d9bc25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:44.525657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] RBAC: Authorizing `identity:get_user(user_id=80a06e2e516e41ecbed6078f06d9bc25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:44.527238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f606e61-58eb-4cdd-94cc-b4ff0f234ddd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:44.530390 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 174/327] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:44 2022] GET /identity/v3/users/80a06e2e516e41ecbed6078f06d9bc25 => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:44.538560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:44.583480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492615-4557-4f9f-9e5f-042caa939a25 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:44.583882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492615-4557-4f9f-9e5f-042caa939a25 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:44.584121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492615-4557-4f9f-9e5f-042caa939a25 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:44.588076 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 175/328] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:53:44.594950 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:44.595332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:44.595593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:44.618918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:44.620139 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:44.723394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 176/329] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:44.758354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:44.778774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:44.794325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fab13-3cc3-4576-93e5-130e833076f3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:44.845333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:44.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:44.846640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:44.847057 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:44.848840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] RBAC: Adding query filter params (role.id=4971f34fbc684f48b7379abfd73b60f8, user.id=80a06e2e516e41ecbed6078f06d9bc25, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:44.849048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:44.850768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:44.873581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 154/330] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 11 11:53:44 2022] GET /identity/v3/role_assignments?user.id=80a06e2e516e41ecbed6078f06d9bc25&scope.system=all&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 510 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:44.923332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:44.977096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2401e-5073-4bd5-9b48-efe1ce83fd51 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:45.055336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:45.056715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:45.057229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:45.057769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:45.079225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:45.095334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:45.096320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:45.097729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:45.098913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 177/331] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:45.105444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:45.363623 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6d48d-c9b3-4f20-a103-c340425945a1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:45.437962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:45.439108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:45.439579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:45.440036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:45.441386 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:45.441927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:45.444449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:45.464033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 155/332] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:45.470526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:45.715945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc0a60-c30a-4ccf-8a12-56999f4c0fa6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:45.783422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:45.785238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:45.785778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:45.786304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] PATH_INFO: `/v3/users/9024cc5d68eb40318890a244a68dc683` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:45.788549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] RBAC: Authorizing `identity:get_user(user_id=9024cc5d68eb40318890a244a68dc683)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:45.790411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e402cf8-613b-4ace-83fd-b247029e985a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:45.793767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 178/333] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:45 2022] GET /identity/v3/users/9024cc5d68eb40318890a244a68dc683 => generated 293 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:45.800130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:45.849015 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:46.031180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975e1e-405d-4462-9086-d77d7237da64 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:46.077614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:46.078940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:46.079457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:46.080001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] PATH_INFO: `/v3/system/users/9024cc5d68eb40318890a244a68dc683/roles/8953d2c24d7c4f8696f1cbfeeae409f0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:46.082458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9024cc5d68eb40318890a244a68dc683, role_id=8953d2c24d7c4f8696f1cbfeeae409f0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:46.084694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5b374-7fff-4787-9d64-4861f1c9c3ce None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:46.107750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 156/334] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 11 11:53:45 2022] PUT /identity/v3/system/users/9024cc5d68eb40318890a244a68dc683/roles/8953d2c24d7c4f8696f1cbfeeae409f0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:47.160976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cfadd-8197-4b2e-9735-6ed9665251ba None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.161997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cfadd-8197-4b2e-9735-6ed9665251ba None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.162552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cfadd-8197-4b2e-9735-6ed9665251ba None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.164187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 179/335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:47.170019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.170502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.170597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.194873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:47.196104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:47.292120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 157/336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:47.340227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200cfadd-8197-4b2e-9735-6ed9665251ba None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:47.343700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:47.484019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200cfadd-8197-4b2e-9735-6ed9665251ba None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:47.531326 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:47.532145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.532429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.532708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.544997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:47.555719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:47.556682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:47.558197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 180/337] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:47.563142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:47.580425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:47.720340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd6b3-7d52-4465-98a9-120039ebe900 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:47.773440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:47.774182 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.774469 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.774807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.775591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:47.775908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:47.777330 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764665a-4dff-4ce7-9da2-1973c992f3df None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:47.788156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 158/338] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:47.795867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:47.796316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6e06-4477-4d79-885b-88ce18fe1a11 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.796617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6e06-4477-4d79-885b-88ce18fe1a11 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.796904 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6e06-4477-4d79-885b-88ce18fe1a11 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.798682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 159/339] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:47.804549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:47.804868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:47.805119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:47.842851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:47.844089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:47.906233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:48.010690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:48.017713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 160/340] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:48.020782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559b322-638c-4eed-9752-04b20fbe5f68 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:48.059611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:48.111281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:48.111984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.112272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.112549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.297981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:48.302169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:48.371318 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5547d7-7589-4824-8ec0-e9701f195603 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:48.447990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:48.449068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.449500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.449964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.459331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:48.478304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0d99cc24-49b6-4529-a298-397416ac05aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:48.487664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:48.488798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:48.489825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:48.499606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 161/341] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:48.500997 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c83b49-c037-4908-a0b1-fba3b551e644 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.501402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c83b49-c037-4908-a0b1-fba3b551e644 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.501837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c83b49-c037-4908-a0b1-fba3b551e644 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.504056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 162/342] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:48.510150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.510553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.511213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.527860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 181/343] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:48.529031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:48.545434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:48.546823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:48.642141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:48.650700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536f0a6-89e5-48ed-9e63-0329e0789a0c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:48.663155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 163/344] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:48.664127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:48.702533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:48.703642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.704144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.704640 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.705733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:48.706270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:48.707962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:48.718224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 182/345] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:48.719415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:48.754099 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:48.873759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a620fa-dddd-42c8-8995-e1c90b3ae35a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:48.907844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552428c-0522-478d-aaa6-772e70ba9bf8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:48.923025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:48.923854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.924162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.925044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.950693 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:48.965947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:48.967853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 183/346] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:48.970102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:48.972092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:48.974435 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:48.974435 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:48.974435 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:48.977741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:48.977741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:48.983595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 164/347] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:48.984451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.113845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.132222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb06f2-3d32-4ad4-b906-7d45f874a994 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.143053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.198588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:49.199192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.199485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.199769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.204322 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e1ebb-a4b5-4380-944b-0e6896b6f019 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.248229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:49.249373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.249916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.250423 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.251404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:49.251922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:49.253595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:49.264000 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 165/348] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:49.265301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.307329 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:49.308849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa1e25-7e9f-4fef-9f02-4fffc1397f97 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:49.385284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2d8ac-497d-4530-bae4-037a59ef465d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.405338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.413713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.427175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 184/349] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:49.433275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:49.434166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adba586-c133-4525-b38e-9ec2e150f5a9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.434638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adba586-c133-4525-b38e-9ec2e150f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.434774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.435144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.435343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adba586-c133-4525-b38e-9ec2e150f5a9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.435617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.437086 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 185/350] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:49.438142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adba586-c133-4525-b38e-9ec2e150f5a9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.444231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:49.445778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:49.447557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 166/351] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:49 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:49.448729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.563329 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adba586-c133-4525-b38e-9ec2e150f5a9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.576388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400a19-2921-4f7b-a864-44231e1bf257 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.616415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:49.617382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.617865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.618318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.621315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:49.622885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.623427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.623919 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.625541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:49.627439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2365987-768e-4e8f-84e9-4d10eb4e6940 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:49.629840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 167/352] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:49 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:49.631501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.632025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.632532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.654063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:49.655312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:49.696860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.716593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:49.717935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:49.718073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398fa34-a470-4868-ae2c-adc73910003a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:49.746221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 168/353] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:49.747462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.863231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 186/354] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:49.866048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e838-a6a4-4125-9e09-e8bf404c5b9c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:49.866654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e838-a6a4-4125-9e09-e8bf404c5b9c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:49.867082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e838-a6a4-4125-9e09-e8bf404c5b9c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:49.869083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 187/355] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:49.870394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e838-a6a4-4125-9e09-e8bf404c5b9c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:49.947060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ad5b0-7737-40fb-8790-d3abb146678e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:49.999761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.000830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.001298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.001862 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.003625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:50.004873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.007312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.008006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:50.016297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 169/356] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:53:49 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.017901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.043687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e838-a6a4-4125-9e09-e8bf404c5b9c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.107097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.108394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.108883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.109344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.111835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.113843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.116518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 188/357] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:49 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.117753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.138468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:50.241480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944152-ab63-47b4-808d-2bf57b4d8ab3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.288138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.289797 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.290250 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.290676 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.305627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-80351da2-8c63-42fb-a50f-8cb784517549 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:50.316265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:50.317237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:50.318082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:50.320311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 170/358] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:50.324529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.324529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.324529 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.329417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1dbc7-2f8b-4f7e-a53f-521be6cc03bf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.347197 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:50.348770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:50.401387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.402374 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.402695 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.402931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.404282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.405982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dfadc-5caa-4055-b5f3-d4c75fbf35cc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.409486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 189/359] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.411287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3195e-380a-4563-bb54-a44ab6eb729c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.411581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3195e-380a-4563-bb54-a44ab6eb729c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.412022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3195e-380a-4563-bb54-a44ab6eb729c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.413837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 190/360] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:50.414771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3195e-380a-4563-bb54-a44ab6eb729c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.459047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:50.468530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 171/361] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:50.470677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.526218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:50.595948 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3195e-380a-4563-bb54-a44ab6eb729c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.624790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22be68d-f131-4f2f-bffa-9b5033596de0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.647032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.647032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.647382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.647755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.648422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:50.648735 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.650219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.657024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 191/362] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.660313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.703842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.705541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.706369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.707151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.708922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:50.710379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.714965 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e746e-4b77-4f92-bf4a-36417ee4e574 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.734139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 172/363] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.737570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.741900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.741900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.778914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:50.781344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:50.798939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:50.822113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87764ad9-7106-43b5-a019-f387a311ca28 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:50.894511 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:50.896030 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:50.896366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:50.896687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:50.900644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:50.901793 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:50.905128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 192/364] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:50.906340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.938324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 173/365] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:50.939300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:50.979399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:51.080008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7324-b7d6-45cb-b715-56e3b4e2b4bf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.133788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:51.135054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:51.135734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:51.139241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:51.145192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1430-4a9d-4064-a903-fa8a3c4b2f8b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.216538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:51.218443 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:51.219247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:51.220032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:51.227223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:51.227223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:51.246112 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:51.256398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:51.263240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:51.264616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:51.266854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:51.271292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 193/366] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:51.271971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:51.298092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:51.381721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 174/367] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:51.381721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:51.475192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe923b-8558-4dc8-8588-01535c5eef8e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.544044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:51.545651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:51.546432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:51.547184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:51.549012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=2c65192a7afb43748595114bfe9ea9bd) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:51.549999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:51.552893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:51.564210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 194/368] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:53:51 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 260 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:51.566244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:51.579663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4eb6a-45d6-420a-beb0-df3ca933310e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.672051 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:51.701881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:51.703134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:51.703603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:51.704060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:51.744441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:51.768131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:51.769945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:53:51.772494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:51.776370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 175/369] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:51.776370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:51.836060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:51.894771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaf903-35b5-43de-9469-e727be55b5b4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.951485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:51.952196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:51.952487 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:51.952767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:51.953550 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:51.953930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:51.955507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:51.960384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae13a-f766-4c59-a1c2-53911c306dbe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:51.973122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 195/370] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:51.973477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.014371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.015323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.015616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.015899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.022739 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.024284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.029789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 176/371] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:51 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:52.029789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.159778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb222b-fa5c-4250-84d9-f5c2d22d3ce2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:52.172952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:52.199214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.200266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.200760 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.201225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.202776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:52.203138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.205040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.215586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:52.223592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 196/372] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:52.223803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.241886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d605c-088b-4b4f-a7c0-1ffe2b5d89d6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:52.297543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.298527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.299250 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.299793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.495440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.497299 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f70fc-30f5-450b-a7ea-1982252f89ab None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.503078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49a6e0-8957-457c-9ff0-c7df2c347e24 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:52.569573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:52.599028 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.599929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.600223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.600502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.606549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.608045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.609991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 197/373] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:52 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:52.610643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.639868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 177/374] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:53:52.644828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92cb4a-6d20-4001-ae0b-1d0790b4cbfa None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.645433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92cb4a-6d20-4001-ae0b-1d0790b4cbfa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.646947 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92cb4a-6d20-4001-ae0b-1d0790b4cbfa None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.649642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 178/375] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:52.652317 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92cb4a-6d20-4001-ae0b-1d0790b4cbfa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.743017 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:52.793122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92cb4a-6d20-4001-ae0b-1d0790b4cbfa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:52.793891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391aec-8e2f-4292-8c5b-26e0739f7840 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:52.838401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.838985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:52.839215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.839691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.840029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.840493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.840885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=2c65192a7afb43748595114bfe9ea9bd) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:52.841070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.841242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.841639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] PATH_INFO: `/v3/users/9024cc5d68eb40318890a244a68dc683` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.842724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728b19f-6609-425d-aab8-03d5b30866fa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.843262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] RBAC: Authorizing `identity:get_user(user_id=9024cc5d68eb40318890a244a68dc683)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:52.844881 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:52.846920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 198/376] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:53:52 2022] GET /identity/v3/users/9024cc5d68eb40318890a244a68dc683 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:52.847977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:52.857714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 179/377] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:53:52 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:52.857714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:52.857714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:52.857714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:52.873726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:52.873726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:52.934116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.055418 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 180/378] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:53.056939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.103141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ae0d0-7a75-46f1-a640-71e3656dafca None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.148551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.149442 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.149781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.150050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] PATH_INFO: `/v3/projects/aeb90a4a47b148ccb3710178a3b17750` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.151894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb90a4a47b148ccb3710178a3b17750)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.154160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.156654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 199/379] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:52 2022] GET /identity/v3/projects/aeb90a4a47b148ccb3710178a3b17750 => generated 344 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.161815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.167140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.219506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6000f-eef6-4a69-b8c2-0365904616e7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.281638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.282362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.282651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.282933 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.284680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] RBAC: Adding query filter params (role.id=8953d2c24d7c4f8696f1cbfeeae409f0, user.id=9024cc5d68eb40318890a244a68dc683, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:53.285119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.286836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.300763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 181/380] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/role_assignments?user.id=9024cc5d68eb40318890a244a68dc683&scope.system=all&role.id=8953d2c24d7c4f8696f1cbfeeae409f0 => generated 510 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.301538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.317109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba39f2-3e57-4843-8803-cb77af684e48 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.321562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.363077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.363820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.364110 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.364390 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.390841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:53.410006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:53.411083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:53.411933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:53.417753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 200/381] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:53.420428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.469170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476c7d-792b-484d-95f0-d6c5ce12f54c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.544747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.545631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.545939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.546215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] PATH_INFO: `/v3/groups/692c7b1141b8499a86c4e84cfe341592` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.555373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.561876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] RBAC: Authorizing `identity:get_group(group_id=692c7b1141b8499a86c4e84cfe341592)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.563390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.565888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 182/382] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/groups/692c7b1141b8499a86c4e84cfe341592 => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.572911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.638483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c099380-dd02-41ae-9ee0-58ef0d946078 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.720413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.721101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.721994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.722279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.723046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:53.723359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.732432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.732718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.746230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 201/383] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.751594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:53.780709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848efc-9836-425b-acb9-8ffa85bcb12a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.824050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.824739 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.825032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.825315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.826741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] RBAC: Adding query filter params (group.id=692c7b1141b8499a86c4e84cfe341592, role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=aeb90a4a47b148ccb3710178a3b17750) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:53.827071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.828542 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea1228-5f47-4e69-8870-de7987965a36 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.836201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 183/384] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/role_assignments?group.id=692c7b1141b8499a86c4e84cfe341592&scope.project.id=aeb90a4a47b148ccb3710178a3b17750&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.838380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c09a-7e5e-4eb8-bc25-072c75b809a4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.838560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c09a-7e5e-4eb8-bc25-072c75b809a4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.838959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c09a-7e5e-4eb8-bc25-072c75b809a4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.840907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 184/385] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:53.850632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.851156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.851603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.874143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:53.875451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:53.916235 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0b8a-02b6-4ede-96af-1e56d3fc6104 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:53.967194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:53.978584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:53.979756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:53.980189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:53.980611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:53.982435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:53.982687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 185/386] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:53.984473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:53.987014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 202/387] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:53.992144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.017950 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.142391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:54.198752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e81fd-36d7-4369-9f21-b6e507859db4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:54.256634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:54.257567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.257857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.258121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.259399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:54.261121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:54.263039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 186/388] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:53 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:54.272053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.301968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f6b52-a76e-4ff3-8a00-26156f59085e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:54.368751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:54.370592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.371357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.372108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.393861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f4801523-2661-4fc3-8d99-6fbbd016de48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:54.410916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:54.412543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:54.414130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:54.418770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 203/389] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:54.421274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9ebfb-64f8-45a3-998d-caa097cc9f5e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.422063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9ebfb-64f8-45a3-998d-caa097cc9f5e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.422672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9ebfb-64f8-45a3-998d-caa097cc9f5e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.425097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 204/390] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:54.429286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9ebfb-64f8-45a3-998d-caa097cc9f5e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.438748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebca57-b7e1-4237-8823-b6e155589179 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:54.505790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:54.535602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:54.538178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.538664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.539897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/groups/df16ec6282ef49fc947aabeb04d223bf/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.546092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, group_id=df16ec6282ef49fc947aabeb04d223bf, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:54.550169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d42cef-e047-4ee7-a23b-593beb2c974d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:54.556142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:54.592780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 187/391] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 11 11:53:54 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/groups/df16ec6282ef49fc947aabeb04d223bf/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:54.595116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.595600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.596084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.597606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9ebfb-64f8-45a3-998d-caa097cc9f5e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:54.634170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:54.636915 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:54.691381 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:54.692128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:54.692417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:54.692709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:54.693541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:54.693938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:54.695533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:54.705708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 205/392] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:54.715604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.778991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 188/393] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:54.867297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:54.969826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e55cbb-b02a-455a-a92b-ec6cb21abb52 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:55.003989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:55.041196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:55.042947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:55.043956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:55.044512 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:55.046618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:55.050875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:55.054623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 206/394] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:54 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:55.067624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:55.085735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94a145-0b01-4e60-af8f-54a112e41431 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:55.183843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:55.217231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:55.218682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:55.219156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:55.219616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:55.256089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:55.287556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:55.288535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:55.289324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:55.295237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 189/395] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:55.302237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:55.321735 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1332a-3905-486a-8418-ffb42b0a8f6e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:55.419702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:55.421208 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:55.423082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:55.423082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:55.424838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:55.427635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:55.431664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 207/396] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:55 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:55.439033 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:55.527005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:55.569525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b0f2-d455-46f2-9d9a-3eab09e758e4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:55.604916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e585b3-355c-4cbe-bd8b-ab28fde00bbe None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:55.688343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:55.692536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:55.692536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:55.692536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/groups/df16ec6282ef49fc947aabeb04d223bf/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:55.693699 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:55.694767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:55.695289 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:55.695714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:55.696349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] RBAC: Authorizing `identity:create_grant(project_id=c68efe47f71b4d57ae94075a4b6a3a63, group_id=df16ec6282ef49fc947aabeb04d223bf, role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:55.696801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:55.697260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:55.698839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:55.699547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:55.720418 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 190/397] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:55.724741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 208/398] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 11 11:53:55 2022] PUT /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63/groups/df16ec6282ef49fc947aabeb04d223bf/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:55.726646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:55.874242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:55.988292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e417d6-39af-48e8-bc9b-a2a9c1ede5a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:56.076458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:56.077930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.078416 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.078893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.080981 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:56.083877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc8c65-89bf-45a5-9cd8-f16d48e9b90a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:56.087416 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 191/399] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:55 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:56.093718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:56.220109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae620-5635-4733-affa-f9ab42330082 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.220740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae620-5635-4733-affa-f9ab42330082 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.221815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae620-5635-4733-affa-f9ab42330082 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.223690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 192/400] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:56.229725 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.230363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.230854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.252080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:56.253769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:56.267342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7c96-0157-48e1-a276-87f0d3fc5c8a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:56.303376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:56.359063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:56.359995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.360275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.360564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.363861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:56.365415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:56.367669 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 209/401] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:56 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:56.376065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:56.427484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 193/402] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:53:56.480828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:56.551221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:56.579089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7240d7-dfa5-47a7-a3a6-0bf7c18da039 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:56.667454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:56.669036 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.669562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.670216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/groups/df16ec6282ef49fc947aabeb04d223bf/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.673906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, group_id=df16ec6282ef49fc947aabeb04d223bf, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:56.676777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce3e7-93d4-4c41-b146-282d12338398 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:56.686058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478718c3-201d-4e77-ae1d-19ef20699315 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:56.706826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 210/403] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 11 11:53:56 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/groups/df16ec6282ef49fc947aabeb04d223bf/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:56.714939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d531-2406-4a1f-91c1-bb94c398b850 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.718364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d531-2406-4a1f-91c1-bb94c398b850 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.718934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d531-2406-4a1f-91c1-bb94c398b850 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.726756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 211/404] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:56 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:56.737962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.738887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.739852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.783624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:56.788564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:56.789521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:56.790267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:56.800274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:56.802440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:56.839178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:56.859642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:56.861341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:53:56.862960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:56.870190 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 194/405] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:56.876772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:56.894186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:57.025855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 212/406] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:57.033576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75d6f-fdf4-4bf8-b9a2-481f20f209c9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:57.036346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:57.065362 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:57.072475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:57.073224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:57.073506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:57.073815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:57.074593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:57.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:57.076314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:57.087112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 195/407] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:53:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:57.099011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:57.232722 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcbaf1-2728-4cb7-9b5c-932ebe8e3f9e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:57.282315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:57.283098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:57.283364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:57.283846 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:57.285721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:57.287215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:57.289471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 196/408] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:57 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:57.299108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:57.341712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c70afe-6671-4684-8d04-b1f553428f49 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:57.428095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:57.496414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:57.497842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:57.498345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:57.498846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:57.557816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:57.583525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:57.585458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:53:57.586884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:53:57.592964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 213/409] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 529 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:53:57.604119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:57.699783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b39b-09b4-4e2b-bd31-5ef6a44e779c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:57.715001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:57.816972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:57.818919 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:57.821272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:57.823398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:57.828141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:57.832588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:57.838598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 197/410] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:57 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 540 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:57.848560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:58.002439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cae5e2-5e0c-42c8-9638-b04929249604 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:58.091263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:58.092805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:58.093552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:58.094319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:58.095859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:58.097469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:58.104271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c59f7-3b2e-4418-85ad-47dda5aced44 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:58.144564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 214/411] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:58.147498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32bbb-e6c1-4769-9aaa-40831be451fd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:58.157479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32bbb-e6c1-4769-9aaa-40831be451fd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:58.158455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32bbb-e6c1-4769-9aaa-40831be451fd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:58.163056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 215/412] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:58 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:58.164944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed32bbb-e6c1-4769-9aaa-40831be451fd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:58.243989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5602dc-6b14-4690-9603-defbf5897d08 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:58.248288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:58.329358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:58.331534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:58.332035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:58.332526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd/groups/df16ec6282ef49fc947aabeb04d223bf/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:58.345525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c65192a7afb43748595114bfe9ea9bd, group_id=df16ec6282ef49fc947aabeb04d223bf, role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:58.348698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6976e9-5a45-4f89-9d20-8fd8c7ad4064 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:58.411765 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 198/413] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 11 11:53:57 2022] PUT /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd/groups/df16ec6282ef49fc947aabeb04d223bf/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:58.413207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:58.413756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:58.414429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:58.491560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:58.493591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:58.495937 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:58.538523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed32bbb-e6c1-4769-9aaa-40831be451fd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:58.659182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:58.661004 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:58.662121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:58.662642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] PATH_INFO: `/v3/groups/692c7b1141b8499a86c4e84cfe341592` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:58.665011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] RBAC: Authorizing `identity:get_group(group_id=692c7b1141b8499a86c4e84cfe341592)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:58.668268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:58.672493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 216/414] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:53:58 2022] GET /identity/v3/groups/692c7b1141b8499a86c4e84cfe341592 => generated 229 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:58.679285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:58.765183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 199/415] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:58.815940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:58.919882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33a95b-4c67-41b0-855d-b889f90bcc96 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:59.026522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:59.027436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.027731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.028011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] PATH_INFO: `/v3/projects/aeb90a4a47b148ccb3710178a3b17750` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.029386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb90a4a47b148ccb3710178a3b17750)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:59.030904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:59.033022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 217/416] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:53:58 2022] GET /identity/v3/projects/aeb90a4a47b148ccb3710178a3b17750 => generated 344 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:59.038806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:59.066496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:59.091233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3792-bf81-487c-99f8-dc63b6d447b2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:59.133209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:59.247091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:59.247950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.248285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.248531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.282195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:53:59.284436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804576-3432-45fd-b15d-2eeee70c68c1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:53:59.297693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:53:59.300036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:53:59.301576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:53:59.305867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 200/417] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:53:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:53:59.310819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:59.370618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:59.372627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.373392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.374126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] PATH_INFO: `/v3/projects/aeb90a4a47b148ccb3710178a3b17750/groups/692c7b1141b8499a86c4e84cfe341592/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.378286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb90a4a47b148ccb3710178a3b17750, group_id=692c7b1141b8499a86c4e84cfe341592, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:59.381344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830d1b7-7c05-42a5-87f4-f760588dc85f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:59.412191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 218/418] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 11 11:53:59 2022] PUT /identity/v3/projects/aeb90a4a47b148ccb3710178a3b17750/groups/692c7b1141b8499a86c4e84cfe341592/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:59.547485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d41d25-3a18-4190-92a0-81604c00ec88 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:59.620911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:59.622276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.622778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.623275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.624685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3ca7-760f-43df-9cba-cd960f6a59ac None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.624878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:59.625151 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3ca7-760f-43df-9cba-cd960f6a59ac None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.625428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:59.628289 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:59.629148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3ca7-760f-43df-9cba-cd960f6a59ac None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.632466 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 219/419] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:53:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:53:59.640280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.640623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.640869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:59.647251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 201/420] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:53:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:53:59.659858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:59.675915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:59.677155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:59.706758 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:59.806070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 220/421] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:53:59.857797 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1d8d-1ac3-4233-8fc1-f787d6995b63 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:59.868784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:59.898684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:59.946036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:59.947155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:59.947655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:59.948093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.125135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:00.128756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:00.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:00.204740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b5be1-6138-493f-a8b8-e4808595502e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:00.342866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:00.344556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.346551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 202/422] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:53:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:54:00.347835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.348531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.359284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:00.370617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:00.387051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:00.389183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:00.391056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:00.395262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 221/423] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:53:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 527 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:00.401986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:00.600643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:00.623539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc2c8-3752-4450-8fb7-6fbb144953b6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:00.651360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6cc92-b73b-4b60-b171-6e64a058bd8d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:00.658588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:00.696238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:00.698110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.698813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.699493 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.701954 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:00.704792 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0fa67-d1ed-4df9-a17a-ce6f4ba330fd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:00.713843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 203/424] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:54:00 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:00.716496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75746071-a8d8-4703-a564-c345941e6656 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.717237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75746071-a8d8-4703-a564-c345941e6656 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.718013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75746071-a8d8-4703-a564-c345941e6656 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.720674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 204/425] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:00.723543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:00.724369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.724836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.725325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.726528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75746071-a8d8-4703-a564-c345941e6656 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:00.726762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:00.727527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:00.730358 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f9987-0301-4553-9735-3f4c8b2fa38d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:00.748464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 222/426] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:00.751255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.751987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.752771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:00.784983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:00.787878 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:00.922557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75746071-a8d8-4703-a564-c345941e6656 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:00.927672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 223/427] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:00.929612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:00.994530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:00.996397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:00.997093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:00.997863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.000302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:01.004224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:01.007883 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 205/428] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:54:00 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:01.015384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:01.077205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:01.132111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:01.164009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649847b-247f-4b88-bfb3-38d2098ecd1d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:01.227552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38bec-1da8-4c5f-a5a3-ae8dda8ba042 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:01.231055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:01.233211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.233994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.234771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.293812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:01.295235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.295945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.296630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.317193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:01.334099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:01.336394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:54:01.338370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:01.342854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 206/429] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:01.344730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:01.375938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:01.378314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:01.505408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:01.536433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a60e7-1cce-4a7c-bbff-5dc0bfcc9a9b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:01.555342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:01.565733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 224/430] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:01.565733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:01.601429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:01.603073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.603535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.604255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.605631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:01.606160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:01.608322 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ba8fb-91df-4444-bc57-be83e9dae0b8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:01.622274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 207/431] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:54:01 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 611 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:01.626163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e42a6-f8de-42a4-a04b-2cf5a3e41b8a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.627723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e42a6-f8de-42a4-a04b-2cf5a3e41b8a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.628538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e42a6-f8de-42a4-a04b-2cf5a3e41b8a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.631921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 208/432] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:01.640351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e42a6-f8de-42a4-a04b-2cf5a3e41b8a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:01.762850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d5ece-b845-450c-990d-a28a570b21da None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:01.842101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:01.843441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.843906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.844333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.845576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:01.846759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:01.849411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52836aab-c0cb-4842-9c1a-7d208b8a52e6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:01.859466 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e42a6-f8de-42a4-a04b-2cf5a3e41b8a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:01.865269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 225/433] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:54:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:01.867483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.867805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.868069 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.905131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:01.908838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:01.925324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:01.926530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:01.926817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:01.927099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] PATH_INFO: `/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:01.928672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] RBAC: Authorizing `identity:get_project(project_id=c68efe47f71b4d57ae94075a4b6a3a63)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:01.930294 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:01.932107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 209/434] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:54:01 2022] GET /identity/v3/projects/c68efe47f71b4d57ae94075a4b6a3a63 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:01.933025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:01.966559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.024968 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.035025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 226/435] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:02.036123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.158315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c6ae-9dd7-4b15-a4fe-ebeec78a839e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.177263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa38fa2-c701-4341-8296-5b59cd4d1ba4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.210048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.210826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.211113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.211394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.227024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.228786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.229244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.229745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.232205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.234637 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.237379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 227/436] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:02.239870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.280510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.318302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.319789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.365268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64307-093d-453d-87de-4a0631d3bd8a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.371352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.404530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.405356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.405640 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.405969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.419303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:02.428181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:02.429174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:02.430012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 228/437] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:02.435637 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.446683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 210/438] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:02.447328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.548342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.561639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344a4ee-e6af-41d1-879c-9791f620b7a3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.571806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb437e-4c25-41b2-b9a7-32757b53880a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.612213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.613139 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.613429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.613880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.615183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:02.615509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.617298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.619750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.620441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.620817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.621017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.621815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:02.622139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.623555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.625642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 229/439] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=c68efe47f71b4d57ae94075a4b6a3a63&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:02.627100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.633135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 211/440] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:02.638686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.644220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.771564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbabe3-3545-4203-9250-81d661d13073 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.810761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739c0a-a589-4ad3-8f72-a6c297af856c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.823985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.823985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.823985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.824620 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.826080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.827713 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05266b2d-9c31-4d08-97ba-c1646eee1f52 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.831486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 230/441] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:02.834333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cddb1-d270-4cb7-a263-d19688cb4ec9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.834720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cddb1-d270-4cb7-a263-d19688cb4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.835183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cddb1-d270-4cb7-a263-d19688cb4ec9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.837654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 231/442] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:02.842774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cddb1-d270-4cb7-a263-d19688cb4ec9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:02.867528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:02.868205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:02.868499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:02.868781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:02.875994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:02.980437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:02.982541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:02.983402 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cddb1-d270-4cb7-a263-d19688cb4ec9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:02.987544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.036376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.037257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.037539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.037846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.039660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.041595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01056814-7a96-442e-8c6a-639749aeae06 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.043355 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 232/443] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:03.046579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.046883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.047255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.066103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:03.067744 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:03.118946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 212/444] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:54:03.119333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.146683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 233/445] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:03.147841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.183973 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.242568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218807a2-428b-4f1d-ac7c-02c3439aec86 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.245968 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.279223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.279900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.280187 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.280467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.281310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=4971f34fbc684f48b7379abfd73b60f8, scope.project.id=2c65192a7afb43748595114bfe9ea9bd) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:03.281630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.283163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.284444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8719-b9e7-4b3c-b3b1-afa1b7da5667 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.290176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 213/446] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=4971f34fbc684f48b7379abfd73b60f8 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:03.291020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.335428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.336236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.336524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.336801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] PATH_INFO: `/v3/projects/2c65192a7afb43748595114bfe9ea9bd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.338124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] RBAC: Authorizing `identity:get_project(project_id=2c65192a7afb43748595114bfe9ea9bd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.339775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.341629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 234/447] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/projects/2c65192a7afb43748595114bfe9ea9bd => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:03.347548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.450580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7330-956c-4ec2-abab-da395b4092dd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.483332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.500253 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.501722 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.501854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149427c6-ea5c-44da-91a2-37ae19207903 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.501959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.501959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.520590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:03.528705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:03.529988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:03.530942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 214/448] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:03.537110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.550739 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.551632 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.551918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.552681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] PATH_INFO: `/v3/groups/df16ec6282ef49fc947aabeb04d223bf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.553514 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] RBAC: Authorizing `identity:get_group(group_id=df16ec6282ef49fc947aabeb04d223bf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.555023 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.556932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 235/449] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/groups/df16ec6282ef49fc947aabeb04d223bf => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:03.563643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.579516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.709069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02a74-1208-4844-994d-7cec48e21b9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.719434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b20b2-fe4b-4f0b-8344-9f56455888a0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.755334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.756010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.756296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.756647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.757354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:03.757667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.759101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.764237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.764920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.765181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.765458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:03.766338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] RBAC: Adding query filter params (group.id=df16ec6282ef49fc947aabeb04d223bf, role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=2c65192a7afb43748595114bfe9ea9bd) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:03.766652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:03.768120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:03.773482 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 215/450] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:03.774871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:03.778907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 236/451] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/role_assignments?group.id=df16ec6282ef49fc947aabeb04d223bf&scope.project.id=2c65192a7afb43748595114bfe9ea9bd&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:03.885230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:03.897252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee579d9-2e69-487d-b325-f11e21afdef4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:03.936876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:03.937537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:03.937860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:03.938142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:04.030041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:04.033009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:04.139630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:04.153142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 216/452] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:54:04.159615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:04.183151 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:04.262339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec20161-df06-4c6a-9655-52746000c31d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:04.297619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:04.298435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:04.298723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:04.299047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] PATH_INFO: `/v3/projects/aeb90a4a47b148ccb3710178a3b17750` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:04.300272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb90a4a47b148ccb3710178a3b17750)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:04.301775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:04.303540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 237/453] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:54:04 2022] GET /identity/v3/projects/aeb90a4a47b148ccb3710178a3b17750 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:04.309304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:04.414221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646280b8-0f40-4130-a6f6-e196e9c1bb54 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:04.449258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:04.450152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:04.450443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:04.450721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] PATH_INFO: `/v3/groups/692c7b1141b8499a86c4e84cfe341592` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:04.451915 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] RBAC: Authorizing `identity:get_group(group_id=692c7b1141b8499a86c4e84cfe341592)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:04.453466 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73e5a-c95b-410b-ba7f-f8b19bcebcee None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:04.455141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 217/454] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 11 11:54:04 2022] GET /identity/v3/groups/692c7b1141b8499a86c4e84cfe341592 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:04.462018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:04.553779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:04.568407 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ed9d9-d371-423a-a1f9-9c4c3c098563 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:04.609456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:04.610171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:04.610462 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:04.610743 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:04.611638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] RBAC: Adding query filter params (group.id=692c7b1141b8499a86c4e84cfe341592, role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=aeb90a4a47b148ccb3710178a3b17750) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:04.611961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:04.613460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57f9f7-9eeb-4e16-a72d-016b313320b1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:04.620423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 238/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 11 11:54:04 2022] GET /identity/v3/role_assignments?group.id=692c7b1141b8499a86c4e84cfe341592&scope.project.id=aeb90a4a47b148ccb3710178a3b17750&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:07.730080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ff6f-5645-4fea-b58a-b3dc21582f30 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:07.731637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ff6f-5645-4fea-b58a-b3dc21582f30 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:07.732414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ff6f-5645-4fea-b58a-b3dc21582f30 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:07.735693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 218/456] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:07.745929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6271-9b03-4210-b9bf-8c5db1c0101f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:07.747332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6271-9b03-4210-b9bf-8c5db1c0101f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:07.748105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6271-9b03-4210-b9bf-8c5db1c0101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:07.785016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ca2d6271-9b03-4210-b9bf-8c5db1c0101f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:07.786708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ca2d6271-9b03-4210-b9bf-8c5db1c0101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:07.886994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 239/457] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:54:07.933882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8ff6f-5645-4fea-b58a-b3dc21582f30 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:08.019853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:08.082084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8ff6f-5645-4fea-b58a-b3dc21582f30 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:08.141022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:08.141876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.142155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.142426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.162400 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:08.163884 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:08.188911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc789-3aea-4122-8b23-9dc808dd173f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:08.189307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:08.190730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:08.191577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:08.191577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:08.191577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:08.191577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:08.191577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 219/458] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:08.192559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc789-3aea-4122-8b23-9dc808dd173f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.193044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc789-3aea-4122-8b23-9dc808dd173f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.195018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 240/459] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:08.201140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:08.203125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a242-a847-4866-be3c-2abb1265ba4b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.203649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a242-a847-4866-be3c-2abb1265ba4b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.204128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a242-a847-4866-be3c-2abb1265ba4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.234064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-70b8a242-a847-4866-be3c-2abb1265ba4b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:08.236709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-70b8a242-a847-4866-be3c-2abb1265ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:08.364620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08aa26-bb27-4def-ab6f-346fd38aa299 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:08.388336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:08.395097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 241/460] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:08.398758 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77388-ea5b-421f-8741-00269ebfbb1b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.399853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77388-ea5b-421f-8741-00269ebfbb1b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.400276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77388-ea5b-421f-8741-00269ebfbb1b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.404238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 242/461] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:08.423931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaab2d0-7d04-4993-8757-4a9dc2f3b56e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.424720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaab2d0-7d04-4993-8757-4a9dc2f3b56e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.425412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaab2d0-7d04-4993-8757-4a9dc2f3b56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.466316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-deaab2d0-7d04-4993-8757-4a9dc2f3b56e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:08.466994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:08.467563 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-deaab2d0-7d04-4993-8757-4a9dc2f3b56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:08.468174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.468679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.469189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.472254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:08.472829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:08.475640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:08.477090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:08.492976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 220/462] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:08.494284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:08.582411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 243/463] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:08.584739 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e13da-84c7-458e-bcea-472a1e127df1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.585243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e13da-84c7-458e-bcea-472a1e127df1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.585649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e13da-84c7-458e-bcea-472a1e127df1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.587137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 244/464] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:08.588545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e13da-84c7-458e-bcea-472a1e127df1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:08.676088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3277b-02d3-45d8-bff4-4992dbbbdf59 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:08.732268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:08.734648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.735353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.736165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.752635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e13da-84c7-458e-bcea-472a1e127df1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:08.756192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:08.758935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-974fef1f-0f70-4ddc-ab8e-67f2b53483c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:08.791117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:08.792042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:08.792892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:08.792892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:08.792892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:08.792892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:08.792892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 221/465] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:08.794660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.795314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.795818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.818391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:08.819346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:08.819663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:08.819921 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:08.820627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:08.822134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:08.836185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:08.838967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:08.845702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:08.858006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 245/466] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:08 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:08.861068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:08.878652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:08.995307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 222/467] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:09.003731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.107020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555082e3-e824-4e9b-bfd6-dce9d9ec3a3f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.183979 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.184822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.185284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.185748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.208430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.210302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.232034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e4da7-d802-488c-a6bb-62e5714c1258 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:09.233822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:09.234731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:09.235686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:09.235686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:09.235686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:09.235686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:09.238564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 246/468] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:09.244515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.310593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:09.310783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.311840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.312364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.312851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.314267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:09.314871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.317605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.345071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 223/469] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:09.345071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.365659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:09.392514 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c918-7a2c-4606-ba15-24db5b0010c4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.458844 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.460549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.461091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.461593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.468001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bc14a-0df4-4e4c-bbd3-a3de3af301e1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.483099 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.485947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.507851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.508534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.508817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.509093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.509867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:09.510178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.511587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ee820774-c5e0-420a-b9f5-d35d736262d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:09.520037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:09.521115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:09.522214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:09.522214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:09.522214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:09.522214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:09.522820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 247/470] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:09.525408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4920356-ea1e-4c6a-9747-55fecde9a4ea None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.525971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4920356-ea1e-4c6a-9747-55fecde9a4ea None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.526481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4920356-ea1e-4c6a-9747-55fecde9a4ea None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.533895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 224/471] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:09.533895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.534192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 248/472] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:09.534192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4920356-ea1e-4c6a-9747-55fecde9a4ea None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.634139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:09.690042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a36bb-d53c-4453-b096-09a285173839 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.698534 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4920356-ea1e-4c6a-9747-55fecde9a4ea None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:09.701548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:09.743089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.744448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.745060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.745579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.747042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.750371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45dce2-e0bf-4272-a15a-f71c0ebe7e2f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.754398 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:09.755179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.755335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.755700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.756335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:09.756641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:09.758067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:09.781296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 249/473] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:09.783978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.797715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 225/474] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:09 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:09.797715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:09.797715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:09.797715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:09.816823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:09.818632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:09.960083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 226/475] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:09.961665 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:09.982607 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d28ca-5b59-40f1-a9a6-6b0639dec1d8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:10.052117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:10.053039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:10.053555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:10.054102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:10.055475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:10.058322 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:10.064157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:10.094373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 250/476] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:09 2022] POST /identity/v3/users => generated 246 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:10.096217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:10.151510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:10.155885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c1567-0208-4db5-85ba-f3fb47a78006 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:10.217798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:10.217798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:10.217798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:10.217798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:10.218217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:10.218217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:10.248606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 227/477] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:09 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:10.324930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557d4e-22e5-4e93-b93e-9febc79ea2d0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:10.373060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:10.373806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:10.374094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:10.374377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:10.375160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:10.376672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:10.388690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.sql.core [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:54:10.388690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 11:54:10.388690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [parameters: {'id': 'c740eb71cc65412eafcceef1c074819d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 11:54:10.388690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80223) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jul 11 11:54:10.412549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return ref.to_dict() Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:54:10.413524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application yield resource Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise self.value Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application session.commit() Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:54:10.414507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:54:10.415412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:54:10.416439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:54:10.417311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [parameters: {'id': 'c740eb71cc65412eafcceef1c074819d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:10.418155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 11:54:10.418977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:10.419828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 251/478] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 11 11:54:10 2022] POST /identity/v3/roles => generated 144 bytes in 320 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:10.428241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:10.620995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba699-a783-4875-a524-3e0301a48751 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:10.652123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:10.693352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:10.694447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:10.694933 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:10.695412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:10.715634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a1649612-4885-4a03-9623-d090bdeddeaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:10.724579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:10.725539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:54:10.726389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:10.729914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 228/479] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:54:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:10.737148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:10.858444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:10.883554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6453f9-d37b-4c27-8b9d-cda2213053fe None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:10.927791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:10.928883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:10.929376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:10.929890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:10.930875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:10.931399 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:10.933031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442a73-2e4a-4df2-b27a-a6f5ccc18c0f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:10.943067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 252/480] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:11.460572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7001-e1d4-4413-bb84-a71df31ca54d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:11.461331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7001-e1d4-4413-bb84-a71df31ca54d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:11.461331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7001-e1d4-4413-bb84-a71df31ca54d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:11.462586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 229/481] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:54:11.468249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01418857-b032-4832-a050-a5d788faab22 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:11.468641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01418857-b032-4832-a050-a5d788faab22 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:11.468895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01418857-b032-4832-a050-a5d788faab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:11.492652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-01418857-b032-4832-a050-a5d788faab22 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:11.496268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-01418857-b032-4832-a050-a5d788faab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:11.603540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 253/482] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:11.628759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7001-e1d4-4413-bb84-a71df31ca54d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:11.777788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7001-e1d4-4413-bb84-a71df31ca54d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:11.823134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:11.823961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:11.824255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:11.824539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:11.837556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:11.846214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:11.847180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:11.848417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 230/483] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:11.853699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01418857-b032-4832-a050-a5d788faab22 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:11.981044 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:12.014098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01418857-b032-4832-a050-a5d788faab22 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:12.057749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:12.059257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.059940 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.060656 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.062125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:12.062907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:12.065909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:12.082219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 254/484] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:12.091539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:12.181417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:12.301151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f5a1c-cae3-4c11-886b-336a90841b9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:12.354541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:12.355296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.355590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.355875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.370188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:12.371762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0bc1bd83-6b33-43b9-b25d-e285099b87db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:12.393854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:12.394786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:12.395624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:12.395624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:12.395624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:12.395624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:12.395624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 231/485] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:12.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:12.586221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaedcf-b2d9-4786-a986-b53b80772922 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:12.612565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffed3-e252-4b8a-93f4-bdd6488140de None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.613087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffed3-e252-4b8a-93f4-bdd6488140de None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.613193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffed3-e252-4b8a-93f4-bdd6488140de None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.614743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 232/486] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:54:12.623578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.623876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.624162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.640288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167d217-e114-459d-8907-c8182342d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:12.641376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d217-e114-459d-8907-c8182342d53a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.641966 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d217-e114-459d-8907-c8182342d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.643004 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d217-e114-459d-8907-c8182342d53a None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.644066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167d217-e114-459d-8907-c8182342d53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:12.644556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167d217-e114-459d-8907-c8182342d53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:12.646336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167d217-e114-459d-8907-c8182342d53a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:12.649621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:12.665166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:12.666674 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:12.682576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 255/487] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:12.687478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167d217-e114-459d-8907-c8182342d53a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:12.844231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 233/488] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:12.881814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167d217-e114-459d-8907-c8182342d53a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:12.887458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:12.925312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:12.939922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:12.940769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:12.942059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:12.942363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:12.957811 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:12.959357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:13.004980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:13.006664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:13.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 256/489] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:13.016605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.079310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db19aa-89f7-4405-b130-cc8eea237ed9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:13.130369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.175603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:13.177568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.178466 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.179276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.201494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:13.204632 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a33e-915b-4e2a-8e8c-2866a69a3727 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-07b9a3df-fed6-4f35-80a0-adac87e00ddc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:13.218807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:13.219998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:13.221079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:13.221764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 234/490] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:13.223800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3453de9-dee3-40a7-aa0d-b23add21ed9b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.224265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3453de9-dee3-40a7-aa0d-b23add21ed9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.224749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3453de9-dee3-40a7-aa0d-b23add21ed9b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.230643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 235/491] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:13.230643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0247a1-d400-4c49-b08e-e3eb5fe6f90b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.230643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0247a1-d400-4c49-b08e-e3eb5fe6f90b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.230643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0247a1-d400-4c49-b08e-e3eb5fe6f90b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.230643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 236/492] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:13.231244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0247a1-d400-4c49-b08e-e3eb5fe6f90b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.255975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:13.256671 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.257124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.257235 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.258140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:13.258469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:13.259978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c859cf-bc9d-41bc-bb47-f6bb54444a41 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:13.300518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 257/493] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:13.301700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600b46-75a9-4d80-a1a9-8a57d6d0d482 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.302005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600b46-75a9-4d80-a1a9-8a57d6d0d482 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.302288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600b46-75a9-4d80-a1a9-8a57d6d0d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.319845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cb600b46-75a9-4d80-a1a9-8a57d6d0d482 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:13.321079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cb600b46-75a9-4d80-a1a9-8a57d6d0d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:13.332066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.367487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0247a1-d400-4c49-b08e-e3eb5fe6f90b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:13.408493 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:13.409511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.410005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.410459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.410560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 258/494] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:13.411575 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:13.412065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:13.412823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.413327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.413863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.414198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:13.435454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 237/495] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:13.436827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.442344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:13.444545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:13.513787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.547093 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 259/496] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:13.548413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.563286 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191c41-e18b-4969-b67d-ba3e0f55e250 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:13.603504 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:13.604558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.604872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.605155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.618531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.723247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:13.726001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:13.744534 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40061ed-c528-4a32-9184-f8f2c737f526 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:13.789386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:13.790341 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:13.790632 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:13.790912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:13.804403 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:13.812475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:13.814284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:13.816220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:13.819200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 260/497] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:13.819572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.883644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.923987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 238/498] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:13.923987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:13.991610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:13.992401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d84ef-3934-4b26-aec8-41e7d3c30623 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.056177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.057515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.057912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.058404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.074388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97441-9fc2-4960-a5c3-e58d7a19602d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.078893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.081156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:14.101000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:14.102665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:14.104266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:14.104266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:14.104266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.104266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:14.106138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 261/499] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:14.107799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.147925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.149375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.149930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.150407 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.167948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:14.181806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:14.183582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:14.185229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 239/500] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:14.186689 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.277772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.299497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f67121-2cca-4743-a575-0939fb94e21c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.336531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.336859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.337215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.337511 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.337828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.338906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:14.339190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.340667 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.341553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817bcc-8d1d-4272-bfe7-8c33a8e3823c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.350150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 262/501] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:14.351039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.390308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.391210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.391621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.392005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.406775 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.408765 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:14.429082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:14.430785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:14.431708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:14.431708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:14.431708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.431708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:14.431708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 240/502] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:14.432444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.489335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ce70d-caf6-4943-955e-ceceac8976f9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.526801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.527479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.527769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.528100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.528779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:14.529091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.530532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efffc4-e45d-4112-be55-56e6badddb31 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.548841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 263/503] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:14.550946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80056d-5348-4d6a-adda-938adb2f80fa None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.551434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80056d-5348-4d6a-adda-938adb2f80fa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.551932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80056d-5348-4d6a-adda-938adb2f80fa None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.554819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 264/504] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:14.556187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80056d-5348-4d6a-adda-938adb2f80fa None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.566537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac1cf1-853f-481d-8f64-4f573d7378d9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.576095 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.604944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.605623 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.606023 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.606236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.606994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:14.607308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.608684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.619842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 241/505] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:14.622068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.639851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.695775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80056d-5348-4d6a-adda-938adb2f80fa None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.737661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.738009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77476764-60be-4f2f-8212-b62eff5abdd0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:14.738960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.739423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.739884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.757599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.760420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.782417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:14.783098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.783505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.783823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.784617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:14.785005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:14.786569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e266e-ea52-4d6f-b2ba-48164928b38b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:14.793121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:14.794244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:14.795182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:14.795182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:14.795182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:14.795182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:14.795706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 265/506] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:14.797876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.802420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 242/507] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:14.804288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:14.804644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:14.805006 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:14.823708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:14.824937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:14.887471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.905492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 243/508] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:14.906979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:14.911699 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:14.949690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381878b9-4821-4235-b875-18539dcf54c0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.005479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.006517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.006862 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.007201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.016380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.018157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:15.037190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:15.038962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 266/509] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:15.040714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.070035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc2d15-ebcc-4250-b949-27c44fd51eca None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.121372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.125223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.125223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.125223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.141647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.144369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:15.166893 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:15.168591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:15.169789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:15.169789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:15.169789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:15.169789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:15.169789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 244/510] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:15.170129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.190361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f21c3-76a2-4f1c-b1c0-f4dfff364d82 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.214535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:15.237746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.238788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.239251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.239697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.240957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:15.241467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.243867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.251242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:15.268611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 267/511] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:15.268611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.310101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4ece0-cc15-4ef4-b9c1-f40e580ce5d4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.349862 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.350672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.350953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.351230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.358941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.360439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:15.376130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:15.377853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 245/512] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:15.380056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.407161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f962-06c5-4b4d-979d-fa08e8068fe7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.456763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.458178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.458697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.459104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.475540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.477703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.496104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d7e-d83f-4950-b09e-ddf4f5e1c296 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:15.499907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:15.501085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:15.502045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:15.502045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:15.502045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:15.502045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:15.502045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 268/513] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:15.503122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.511754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:15.544505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.545667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.545957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:15.546218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.546890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.548411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:15.548989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.551459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.594333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 246/514] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:15.595152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.643903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab825d-23a4-4ddb-bbad-d82920a41e44 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.683673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.684368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.684665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.684946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.685742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:15.686062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.687496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.700886 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 269/515] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:15.701798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.754306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7be230-fc07-4f72-90ba-2a663b9e774f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.801190 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.802044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.802333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.802614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.811181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.812693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.816186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:15.826254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef43d5-8831-4efb-ad9f-75e75f3d43dd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:15.845036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:15.846832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:15.848366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 247/516] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:15.850672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.869796 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:15.870499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:15.870771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:15.871052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:15.871864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:15.872177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:15.873641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:15.895284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 270/517] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:15.896171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:15.913974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.001023 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62c72-9733-4936-bd47-37eb908c4eee None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.014826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09248-3cca-463b-ae56-c381fb048181 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.043618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.044302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.044589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.044877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.045616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:16.045961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.047377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.060900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 248/518] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:16.061797 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.063423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.065319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.067133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.067659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.104445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.176793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.178586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.180346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.188551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b86f88-723c-4a78-9305-7880ec86b6ce None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.236680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.237120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.237634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.252907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.255727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:16.283627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:16.284804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:16.285916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 249/519] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:16.286985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.295756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 271/520] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:16.296767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.356499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.424657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d476-7e92-451c-bcfc-6ae10f61fec1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.438669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fb047-6359-4672-a1c6-2f0b7850bc85 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.460454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.461125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.461410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.461711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.462559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=55a5560632ed452c8287a7c91b57e420) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:16.462876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.464328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.473063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 272/521] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/role_assignments?user.id=55a5560632ed452c8287a7c91b57e420&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:16.475001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.494023 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.494963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.495377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.495784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.497276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:16.497783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.499811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.524241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 250/522] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:16.525796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.546572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.682680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140492-43c6-4ecb-83d5-aa423a967b43 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.703197 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62540f7a-b3d7-4a65-9f5d-46e2eb6acf4c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:16.713124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.745898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.747366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.748519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.749364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.750519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.753566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.757650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:16.758503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:16.758868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:16.759217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:16.760213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:16.760601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:16.762276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:16.784321 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 273/523] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:16 2022] POST /identity/v3/users => generated 247 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:16.786221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.809711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 251/524] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:16.809711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:16.960958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:16.988689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d084c2-f2f7-440b-92c1-2322144ef4a1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.021737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba2dd-6d5f-40a9-b1d1-a5ee578889e9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.040568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.041394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.041725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.041995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.062074 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.064705 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.076736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.077415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.077746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.078068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:17.098456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:17.100157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:17.101864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:17.101864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:17.101864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:17.101864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:17.101864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 274/525] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:17.102409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:17.148245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.221085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.222038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.282430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68d8be-97ff-4ab0-bc90-1ad9750d6f02 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.296519 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.344844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 252/526] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:17.346035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:17.357529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.359903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.360730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.361507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.462142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca646-e2e4-4db4-9b69-ea1c91120153 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.499858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.500820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.501194 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.501546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.503103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:17.503459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.504973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.513634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.516999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.524103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 253/527] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:17.524425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:17.556278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.571847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.643112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820a2a-825d-4fba-bec4-916bb808b4f7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.701145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.703061 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.703783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.704475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.732069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.735165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.738342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 275/528] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:17.740200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:17.777347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:17.779905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:17.781991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:17.781991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:17.781991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:17.781991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:17.790604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 254/529] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:17.793761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:17.925718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979983db-dfb4-4809-a473-31d1a32dc000 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:17.956492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.957004 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:17.971505 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:17.972370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:17.972688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:17.972975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:17.981990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:17.983911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:17.997436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3217f-1fee-47cb-b88f-3521ea20648a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:18.001819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:18.003547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.005270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:18.009849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 276/530] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:18.010829 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.069804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.071950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.073323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.074473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.099979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.104285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:18.146436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:18.147977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:18.149579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:18.149579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:18.149579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:18.149579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:18.149579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 255/531] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:18.157273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.203001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed5e8-94c7-42d2-b8fc-702e07c9089a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.255045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.256960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.257723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.258475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.259813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.260385 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.262686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.282230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 277/532] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:18.284954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.300708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ca0e0-7cee-4295-89bd-286a3ab4d1c9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.336489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:18.342498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.343168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.343455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.343730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.344530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.344840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.346947 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.360945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:18.374242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 256/533] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:18.375305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.423382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c032e-08fc-4f0a-bbe8-890b1b19bc8a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.461501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.462599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.463086 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.463563 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.464496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.465009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.466668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.479829 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 278/534] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:18.481012 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.494833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1aa72-65c6-444a-af33-009d800aaffe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.535268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.536090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.536377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.536629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.544485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.546136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:18.561860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:18.562813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.563662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:18.568882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 257/535] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:18.570056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.605745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ae973-90b9-4ce5-adb6-d928aa835019 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.608709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:18.633420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:18.646530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.647356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.647747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.648128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.649079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=b5a31865c8ed4ab78bfdb17780b61d41) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.649517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.651111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.661906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 279/536] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/role_assignments?user.id=b5a31865c8ed4ab78bfdb17780b61d41&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:18.663047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.695866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6ed9-ca8f-472a-b049-5484b82401ae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.738868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.740000 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.740433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.740853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.749607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.752323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:18.770576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:18.771657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:18.772685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:18.775059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 258/537] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:18.781390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.910531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bddeab-77b5-461d-a6a2-80ea5dee36c2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.924173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8be6-1fce-43d6-8f41-ba8724125e30 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:18.926524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:18.951599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.952601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.953034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.953454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.954540 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.955669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.957314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:18.971547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 259/538] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:18.977802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:18.991264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:18.992494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:18.993335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:18.993690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:18.995373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:18.996035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:18.998352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:19.037013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 280/539] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:19.054162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:19.101381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885caba-7643-4f4a-99b0-213c071a4786 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:19.144146 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:19.145111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.145577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.146078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.147378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:19.147892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:19.149526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb906-da64-4b20-b1fa-c45f6a883314 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:19.156668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 260/540] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:18 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:19.167187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:19.318214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336c62-1d15-4b8c-9613-cf440bbc189e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:19.414546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:19.416007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.416475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.416943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.418074 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=0cf7137aab9d4ebb8b83677cc4b0d817) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:19.418583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:19.420219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99d9e7-00fb-47b3-9f1a-2aaaa22968f4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:19.427071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 281/541] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:19 2022] GET /identity/v3/role_assignments?user.id=0cf7137aab9d4ebb8b83677cc4b0d817&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:19.498419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5f46b-28f3-4be9-ac74-ae4b7c1ae8db None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.499006 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5f46b-28f3-4be9-ac74-ae4b7c1ae8db None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.499466 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5f46b-28f3-4be9-ac74-ae4b7c1ae8db None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.501106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 261/542] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:19.507754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9713-1058-40ae-a6d7-93a893d5e26a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.508312 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9713-1058-40ae-a6d7-93a893d5e26a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.508767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9713-1058-40ae-a6d7-93a893d5e26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.539210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5ade9713-1058-40ae-a6d7-93a893d5e26a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:19.540807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5ade9713-1058-40ae-a6d7-93a893d5e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:19.634853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 282/543] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:54:19.662062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5f46b-28f3-4be9-ac74-ae4b7c1ae8db None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:19.717906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:19.787658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6569-35cf-4b43-93e0-0229dd6dddd9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.788015 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6569-35cf-4b43-93e0-0229dd6dddd9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.788345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6569-35cf-4b43-93e0-0229dd6dddd9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.789946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 283/544] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:54:19.795101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.795396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.795683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.805974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5f46b-28f3-4be9-ac74-ae4b7c1ae8db None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:19.833227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:19.837020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:19.872264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:19.885000 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:19.886213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:19.886876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:19.888646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:19.912164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:19.927274 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:19.928271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:19.929074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:19.933567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 262/545] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:19.942732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:19.996871 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 284/546] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:20.039124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:20.056495 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:20.068499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b15e-9cb3-4956-8c1a-ddfd29312822 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:20.186402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:20.187113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:20.187449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:20.187728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:20.188657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:20.192078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:20.193555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:20.216394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 263/547] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:20.226755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:20.260955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a403e-1e7e-4e28-a813-95257e99cf72 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:20.303898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:20.305083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:20.305393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:20.305689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:20.319342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:20.328938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:20.330067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:20.331096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:20.331909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 285/548] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:20.339703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:20.358393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:20.469117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfafe87-7a7d-44e0-afeb-e53a6e0f6b74 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:20.516272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:20.517131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:20.517131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:20.517548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:20.518241 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:20.518560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:20.520003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:20.524592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d94c5-1fb7-45b1-a01f-cb66fc778240 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:20.538333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 286/549] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:20.550284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:20.654969 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:20.655792 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:20.656082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:20.656366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:20.682863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:20.684364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:20.702997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:20.726554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:20.727511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:20.728443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:20.728443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:20.728443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:20.728443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:20.733802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 264/550] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:20.735885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:20.738992 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe154b2-e406-4763-bf96-4b3e86f5469c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:20.768453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:20.788450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:20.790119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:20.794551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:20.795113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:20.818581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:20.821431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:20.868364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:20.869506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:20.870433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:20.870433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:20.870433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:20.870433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:20.875045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 287/551] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:20.882359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.011948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b5e8f-ab07-493f-85e5-7ab901dbab0d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.025733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767681-d430-47de-a98e-9a88530564bd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.060974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:21.085375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.086096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.086381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.086659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.087195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.087390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:21.087707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.088522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.089021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.089150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.089631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.090606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:21.091151 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.092779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.103916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 265/552] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:21.110771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.112605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 288/553] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:21.129503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.197841 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:21.257306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76028fd5-f6c4-4f3f-81d5-d7aa86d6de02 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.326940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.327692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.327978 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.328259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.336852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724ffd-1536-40b4-8698-3b6ff197d163 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.344984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.346654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:21.381663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:21.382701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:21.383648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:21.386300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 266/554] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:21.390614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.391474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.391769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.392053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.393831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.405916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.407450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:21.427113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:21.430168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:21.433159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 289/555] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:21.444618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.505712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:21.591735 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f4252-5560-4723-91c6-06df9f789e71 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.628956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4991c4-99b6-4539-bea5-20ad757fcd6c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.647413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.648659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.649059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.649346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.650261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:21.650583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.652078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edbde2-b192-4978-907f-3a08a0836625 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.676865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 290/556] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:21.676865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e4b09-467b-4abb-a178-62c43fe49ea1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.676865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e4b09-467b-4abb-a178-62c43fe49ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.676865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e4b09-467b-4abb-a178-62c43fe49ea1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.676865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 291/557] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:21.677799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca79f7-c604-4b35-8990-755742739ef2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.677799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca79f7-c604-4b35-8990-755742739ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.678121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca79f7-c604-4b35-8990-755742739ef2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.681827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 292/558] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:21.683535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca79f7-c604-4b35-8990-755742739ef2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:21.691084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:21.700729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.701905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.702450 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.702896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.704480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:21.705058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:21.707768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aed8f-e561-411e-99fd-600bddfed382 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:21.731011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 267/559] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:21.732064 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0fa56-4222-4e0d-8c7f-03ce9a8f0181 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.732378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0fa56-4222-4e0d-8c7f-03ce9a8f0181 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.732646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0fa56-4222-4e0d-8c7f-03ce9a8f0181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.755122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7bd0fa56-4222-4e0d-8c7f-03ce9a8f0181 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:21.756342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7bd0fa56-4222-4e0d-8c7f-03ce9a8f0181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:21.841718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 268/560] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:21.841718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.841718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.841718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.863500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca79f7-c604-4b35-8990-755742739ef2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:21.867689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:21.879002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:21.883099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:21.914349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:21.915009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:21.915292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:21.915571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:21.985553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 269/561] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:21.985553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.054453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.060523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.062041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.133130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30145df6-26cd-4537-8c67-40f9a17d1f14 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.173664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.174506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.174785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.175060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.187590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.189119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:22.210669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:22.218331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 270/562] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:22.220113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.242107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 293/563] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:22.243038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.248191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.346027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.349462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc177b4-3721-478f-8934-323e1b3a65d0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.364019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2466-13a5-4e42-9d1b-965adbcab966 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.399663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.400387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.400678 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.400957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.409427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.410305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.410597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.410880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.412756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-593c091e-fb77-4602-beb3-c271eb10f2b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:22.425827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:22.427756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 294/564] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:22.429529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e3cd-92a0-47af-8614-41354764a9cc None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.435901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e3cd-92a0-47af-8614-41354764a9cc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.435901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e3cd-92a0-47af-8614-41354764a9cc None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.435901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 295/565] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:22.435901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3e3cd-92a0-47af-8614-41354764a9cc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.437906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:22.446979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:22.449277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 271/566] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:22.450928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.525042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.571900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3e3cd-92a0-47af-8614-41354764a9cc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.585580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546143c-97e4-4bf6-9c35-91955223f2de None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.618755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.619535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.619953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.620383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.621244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:22.621663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.623361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.625059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.625913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.626206 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.626491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.637510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 296/567] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:22.638416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.644364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.647277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-dcc91ccc-d6db-444d-9ef2-fe816f3e1ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:22.680972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:22.682943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:22.684546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:22.684546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:22.684546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:22.684546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:22.684546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 272/568] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:22.685569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.686124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.686634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.716332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:22.718413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:22.732848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.771543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2821-b08c-4335-bd16-9431a386add9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.798386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:22.806881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 273/569] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:22.807834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.818905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.819683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.820313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.820429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.821265 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:22.821615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.823880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.834985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 297/570] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:22.838648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:22.941118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999cc07-c83c-4c4e-b772-173c949a9285 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.954350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1585-6279-4482-95aa-ec43796507f4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:22.979609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.980349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.980769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.981151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:22.981950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:22.982455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:22.984652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:22.993214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:22.994048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:22.994349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:22.994613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.000223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 274/571] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.002391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.002560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.003872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.010851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:23.030308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:23.032275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:23.033865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 298/572] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:23.034467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.087650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.134337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ef9f1-ed89-43cb-81b4-8983cdf496e2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.146108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85dc3-f8a5-4909-8141-913a526d6aaa None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.176175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.177320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.177872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.178383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.179681 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:23.180247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.182692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.189062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.191064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.191629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.192087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.203317 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 275/573] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.204287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.210994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.213745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:23.244612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:23.245590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:23.246539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:23.246539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:23.246539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:23.246539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:23.246539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 299/574] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:23.246826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.282230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.325365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3acf6b-eea9-4e3f-984c-1095a9dfee9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.364022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e13488-b28d-4955-a625-5839910a595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.364835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13488-b28d-4955-a625-5839910a595f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.365126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13488-b28d-4955-a625-5839910a595f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.365409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13488-b28d-4955-a625-5839910a595f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.366813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.371994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5455-7992-4b02-82ce-67e5a63334f1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.376622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13488-b28d-4955-a625-5839910a595f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-96e13488-b28d-4955-a625-5839910a595f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:23.384696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:23.386290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 276/575] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:23.387985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e13488-b28d-4955-a625-5839910a595f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.421888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.422693 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.422984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.423242 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.434870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.436352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:23.455612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:23.457159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 300/576] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:23.458914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.518178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e13488-b28d-4955-a625-5839910a595f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.545412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.559194 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.559875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.560159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.560441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.561252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:23.561569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.563075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.582816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a25428-9264-4be9-a038-fce0e22fef49 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.584252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 277/577] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.585401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.629065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.629913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.630194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.630469 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.638270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.639901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:23.655187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:23.656304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 301/578] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:23.657228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.670546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.700498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3008c-f22b-41e6-a4ea-45aa8e0e66a2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.734910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.735479 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.735767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.736588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.737327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:23.737639 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.739065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.752237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 278/579] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.753103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.763672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af94dc-ab25-4f66-8f35-90cdf249d7b1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.802776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.803450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.803741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.804020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.804792 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:23.805095 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.806523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.815603 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 302/580] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.816430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.826359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.902701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6445c2-70dc-4229-8ad3-287fe5be6845 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.911660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:23.942796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.943470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:23.943756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:23.944034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:23.945607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:23.946281 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:23.947124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458b580-a5c5-41f1-8a9b-47f1bb7ae10b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:23.947812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:23.963948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 279/581] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:23.965179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:23.998947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:23.999983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.000341 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.000691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/55a5560632ed452c8287a7c91b57e420/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.017050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=55a5560632ed452c8287a7c91b57e420, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.018547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.035658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 303/582] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:23 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/55a5560632ed452c8287a7c91b57e420/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:24.036567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.116121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6399e-688a-40e2-833f-8aef0c564fe0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.124567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.184847 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.186101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.186618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.187097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.188547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:24.189104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.192058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.201013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d28f7-b039-47fc-950d-374ec3d26ae1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.218352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 280/583] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:24.219958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.240464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.264729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.266141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.266592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.266929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.279184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.281742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:24.308157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:24.309185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:24.310183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:24.310746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 304/584] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:24.311748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.384014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d957ed-0131-4fed-9fef-bedbe4b40e4f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.427558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.428372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.428656 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.428940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.442002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.443491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:24.462776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:24.463732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:24.464618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:24.464618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:24.464618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:24.464618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:24.464618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 281/585] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:24.464896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.483033 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4b97-2089-4aed-9ea1-bf78b7a45839 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.491200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.525484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.526224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.526519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.526789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.539276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.540775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:24.559722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:24.560773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:24.561748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:24.562995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 305/586] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:24.564078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.628322 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.636412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a713-8146-4a87-8020-811701dfed1f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.683568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.685569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.686070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.686523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.687595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=2ca0990c5479452da837df05b9fbcacb) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:24.688075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.690377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.705746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 282/587] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/role_assignments?user.id=2ca0990c5479452da837df05b9fbcacb&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:24.705746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.741554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4ae1-7b45-48ea-9da4-6838c454b76d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.792482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.793070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.793358 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.793635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.794683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:24.795005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.796532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.823779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 306/588] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:24.824820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:24.843910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.879977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482597fe-036f-4d78-986d-3d40b948b6a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.947062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:24.948824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:24.949243 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:24.950483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:24.951212 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:24.953346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:24.954122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ad45-df39-4440-8f34-d6a209d3dffa None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:24.954260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:24.958478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:24.978198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 283/589] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:24.979572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.037890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.039515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.040037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.040591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.042633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:25.043760 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.047126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.091592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 307/590] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:25.093240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.204072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a592-98b3-4c07-80f7-3729bb1dd32c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.243628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.244577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.244868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.245150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.258300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.259938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:25.280460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:25.282364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 284/591] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:25.284709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.289665 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:25.313115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc80f8-c65e-4df1-8e41-332d761ba971 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.384305 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.386060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.386576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.387223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.402291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94b650-a6f0-4c2f-9d66-d0f2b32ee16f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.402990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.406178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.440130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:25.440454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:25.442527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:25.444936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:25.446071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.446071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.446359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 308/592] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:25.446508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.446647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.457444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.459112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:25.479245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:25.481339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:25.483596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:25.483596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:25.483596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:25.483596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:25.484115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:25.489953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 285/593] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:25.489953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.550772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:25.616016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86fe9-3b9a-49e7-8df5-332b79142f94 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.655025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.656193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.656384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.656875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.658525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:25.659042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.660641 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.675338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d230ec7-7258-4807-ab73-da2dd2daf1b3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.682633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 286/594] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:25.683694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.750671 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.752248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.753079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.753812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.756554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:25.757441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.760394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.790262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 309/595] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:25.792129 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.804398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e6c5-d90a-487d-96a5-2b2d6d018661 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:25.806372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:25.846771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:25.847632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:25.848020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:25.848907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:25.849372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:25.849815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:25.851356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:25.870879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 287/596] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:25.870879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:25.959260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:25.996605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e656f-b7fb-4750-8789-bde77afe76c2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.009179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1314c6-67fe-4360-8a02-920f73dc61f7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.043305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.043971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.044264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.044544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.053630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.055021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:26.071443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:26.073427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:26.075424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:26.077397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 288/597] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:26.078944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.087272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.089113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.089941 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.090645 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.105637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:26.113820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.116987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:26.159214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:26.161152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:26.162938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:26.162938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:26.162938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:26.162938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:26.166195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 310/598] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:26.168059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.195611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804a55f-6f2a-4a63-abba-65d6f047a8a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.235939 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.237330 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.237564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.237864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.246087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.247573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:26.262754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:26.264492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:26.266181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:26.272126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 289/599] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:26.272126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.370716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65727f-12fa-43fb-8e9a-bc132ae528f1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.437902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:26.456230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:26.456497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.457915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.458115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.458755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.462483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:26.462934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.465736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.491453 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 311/600] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:26.493094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.507723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393dde44-7d4a-47c8-b827-a5e218a36925 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.577371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.578588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.579152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.580249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.581650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:26.582216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.585216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.617308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 290/601] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:26.617308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.736190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c68e-8ae8-4854-86da-5eefaf125810 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.791290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.791963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.792260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.792708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.793332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:26.793649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.795147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.809260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 312/602] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:26.814690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.833406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c091-db14-45d0-9874-00084d160e2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:26.876366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:26.877311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.877606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.877986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/b5a31865c8ed4ab78bfdb17780b61d41/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.894058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:26.898541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=b5a31865c8ed4ab78bfdb17780b61d41, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:26.900109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c207f0-ca49-4710-9ad8-7f58e35b1adb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:26.905842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:26.920985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 291/603] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:26 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/b5a31865c8ed4ab78bfdb17780b61d41/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:26.922412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba233-4548-40ab-8a96-ffeff73c6602 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:26.922717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba233-4548-40ab-8a96-ffeff73c6602 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:26.923007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba233-4548-40ab-8a96-ffeff73c6602 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:26.924650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 292/604] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:26.927116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ba233-4548-40ab-8a96-ffeff73c6602 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:26.984945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba25df6-7a99-4b2e-970f-2d72f16cb274 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:27.038583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:27.039477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.039713 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.039988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.052294 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:27.053904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c472eb14-07df-4464-bc68-6b9bba1f4827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:27.070960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:27.072711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 313/605] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d14e-2c8d-40c8-a967-a543898489bc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.075010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d14e-2c8d-40c8-a967-a543898489bc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.077768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d14e-2c8d-40c8-a967-a543898489bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.094574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8c89d14e-2c8d-40c8-a967-a543898489bc None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:27.095863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8c89d14e-2c8d-40c8-a967-a543898489bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:27.106479 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ba233-4548-40ab-8a96-ffeff73c6602 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:27.163083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:27.164207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.164673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.165097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.192145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=80890a6017284bf090892fbcbb618f7f, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:27.193542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 314/606] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:27.194948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d7de1-2e75-45d5-b1cd-4c08ffcd2f52 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:27.195097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385518-8f17-4177-a63a-3958d5626ce7 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.195276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385518-8f17-4177-a63a-3958d5626ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.195529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b385518-8f17-4177-a63a-3958d5626ce7 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.197145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 315/607] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:27.197937 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b385518-8f17-4177-a63a-3958d5626ce7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:27.224611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 293/608] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:26 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:27.226918 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.227506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.228021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.256585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:27.258538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:27.277501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:27.370676 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:27.409824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b385518-8f17-4177-a63a-3958d5626ce7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:27.463454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 294/609] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:27.465958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:27.491102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:27.493763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.494934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.495704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.497967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:27.498853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:27.503287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:27.524170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 316/610] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:27.525647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:27.715736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e47ad-dfa0-4a3e-a4b4-7ede8bb491eb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:27.723746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:27.731363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536439-074c-43af-89fa-430e79a171e3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:27.797161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:27.799461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.799920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.800390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.813459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:27.816983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:27.817945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:27.818733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:27.821756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:27.839059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:27.840977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 295/611] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:27.843531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:27.849117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:27.869404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:27.871369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:27.873131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:27.878363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 317/612] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:27.880401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:27.926429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:28.078115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cc5-b10f-423b-bf59-9c6e1ee3d1c6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.094989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500c03a-94c0-4190-a552-d34967d2151e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.151445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.153983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.154793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.155611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/0cf7137aab9d4ebb8b83677cc4b0d817/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.170462 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.171606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.172118 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.172623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.174666 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:28.175221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.177856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.190758 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=0cf7137aab9d4ebb8b83677cc4b0d817, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.193986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.194777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 318/613] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:28.196296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:28.220916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 296/614] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:27 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/0cf7137aab9d4ebb8b83677cc4b0d817/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:28.223130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:28.269715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:28.407755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:28.450130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a07f8-24b9-4b2a-9b76-dab6eea6974d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.524520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2961-eb46-47d6-9040-6a505b8f5699 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.560355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.561850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.562205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.562672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.586414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.588999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.610513 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.612308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.613478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.614429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.616106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:28.616901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.619925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:28.631164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:28.633125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:28.635177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:28.635177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:28.635177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:28.635177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:28.639563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 297/615] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:28.646974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:28.647370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 319/616] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:28.654287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:28.814134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:28.821723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8fec4-fb31-44e9-b61f-5f3c91d08627 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.864184 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.865176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.865566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.865983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.877838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.879499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.892820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9517d-381c-4ebb-9594-03536b2f2076 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:28.913140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:28.914912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:28.916749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:28.916749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:28.916749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:28.916749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:28.922425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 320/617] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:28.930712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:28.931440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:28.932129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:28.932418 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:28.932700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:28.933456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:28.933818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:28.935234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:28.936987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:28.960314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 298/618] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:28.971345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.078749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b0071-f227-48e1-b269-79465c7e6878 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:29.203767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:29.205861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.206598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.207527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.209154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:29.212133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:29.215106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:29.222082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:29.255197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 321/619] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:29.264681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.272789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202986c-f3fe-4695-86c0-9da9707cbab3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:29.354127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:29.355308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.355748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.356289 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.365513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:29.367262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ba71e74c-1b2e-4000-b950-af7563a6979b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:29.385332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:29.386392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:54:29.387355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:29.394578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 299/620] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:29.396442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f6f06-4c33-4af9-8ec1-db59328f7d83 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.396897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f6f06-4c33-4af9-8ec1-db59328f7d83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.397332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f6f06-4c33-4af9-8ec1-db59328f7d83 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.399075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 300/621] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:29.400037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f6f06-4c33-4af9-8ec1-db59328f7d83 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.475439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8d888-3d71-4bd3-928f-7fe1f7f069bb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:29.502007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:29.541661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:29.543302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.544097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.544772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.572029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f6f06-4c33-4af9-8ec1-db59328f7d83 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:29.573210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:29.576573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c23bafcf-d6f8-4468-9ff0-ea9dd3741d99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:29.607409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:29.609425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:29.611779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:29.612821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:29.612821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.612821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.612821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.613303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:29.613485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:29.615018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:29.616767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 322/622] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:29.619497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.620260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.620945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.641818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 301/623] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:29.641818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.661589 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:29.664205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:29.766270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:29.810213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 323/624] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:29.812281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.827812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f4e2a-5d7f-4769-af6c-dc11ad571ba6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:29.854838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:29.900694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:29.902073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:29.902713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:29.903300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:29.905105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:29.905658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:29.909114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:29.942266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 302/625] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:29.943925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:29.984805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1ba61-0642-4e68-960d-5cde5086d6db None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.036354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.037297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.037757 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.038175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.118734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:30.123353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43baf436-dd41-4912-8c2e-3f9b10ae68f3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.146747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:30.149384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:30.195079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.196953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.197774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.198560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.221933 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-914dcd9e-0855-4f7a-8669-3c01d1b38e6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:30.240406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:30.242314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:30.243823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:30.248050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 303/626] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:30.250892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a84a-2898-482d-a475-acaf772e43c4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.251438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a84a-2898-482d-a475-acaf772e43c4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.251936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45a84a-2898-482d-a475-acaf772e43c4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.254721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 304/627] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:30.256358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45a84a-2898-482d-a475-acaf772e43c4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.287015 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 324/628] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:30.289554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.316541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:30.417377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:30.419022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45a84a-2898-482d-a475-acaf772e43c4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.447818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b1bbb-67ab-48bf-a6fc-4f3c9f2810ab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.461319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.462162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.462441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.463285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.476886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:30.478403 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8a5fc6b6-2e6e-43b8-bd53-78faba6976bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:30.497387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:30.499152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:30.500898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:30.500898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:30.500898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:30.500898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:30.503308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 305/629] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:30.504677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.504974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.505260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.516794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.518888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.519608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.520104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.521522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:30.522109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:30.523815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:30.524703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:30.525028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:30.541969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 325/630] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:30.543569 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.612842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 306/631] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:30.614089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.640337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:30.737386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0d906-4573-4009-8082-fee1754e07e9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.745775 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40616e5e-4b26-4213-8208-e7af0af5fa3c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:30.793143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.794387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.794855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.795310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.796616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:30.797237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:30.799844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:30.803175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:30.804536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:30.805030 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:30.805531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:30.822085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 307/632] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:30.823658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.828315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:30.830831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:30.840627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:30.854024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:30.855752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:30.857463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:30.857463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:30.857463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:30.857463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:30.857936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 326/633] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:30.858863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:30.950367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.006191 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2f866-c199-4391-8110-6787e24ff15b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.014816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9aa1bc-a152-4805-bf95-41cb500b365e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.053120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.054556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.055275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.055853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.074053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.076668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.078868 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.080496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.080961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.081419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.102556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:31.102815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:31.104474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:31.106261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:31.106261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:31.106261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:31.106261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:31.106261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 308/634] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:31.107520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:31.125434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:31.127951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 327/635] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:31.129584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.225230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.254578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cec671-ce08-44fc-8118-e4fc675ddbb8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.284095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.285060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff16f-cf53-48d4-80ad-0f6dd9248a13 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.296366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.297174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.297461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.297766 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.305751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.307233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:31.322999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:31.324659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 309/636] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:31.325753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.332518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.333229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.333499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.333817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.334828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:31.335140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.336542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.349989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 328/637] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:31.350852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.461289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b44b8-e419-4002-9882-70a13d889790 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.484445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.492202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce11a6-d79a-489e-ab31-01066a9610fa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.508399 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.509179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.509525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.509899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.510723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:31.511087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.512602 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a3ccd-bb0a-46f3-8a67-4bcb8cdaf15d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.521542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 329/638] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:31.523194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d042c-8c0a-4792-8205-86eb8ddf158d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.523600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d042c-8c0a-4792-8205-86eb8ddf158d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.523991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d042c-8c0a-4792-8205-86eb8ddf158d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.525662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 330/639] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:31.526601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d042c-8c0a-4792-8205-86eb8ddf158d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.544890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.545867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.546269 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.546687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.547538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:31.547971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.550125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.562215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 310/640] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:31.563080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.622496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.665868 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d042c-8c0a-4792-8205-86eb8ddf158d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.693823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cc598-29da-4417-bfa6-b1ce50552b09 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.715691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.716764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.717110 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.717543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.718955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:31.719462 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.722080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.738425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 331/641] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:31.739997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.742650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.743638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.744075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.744499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.754800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.756730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-de01ab56-8bcb-4762-b625-5da4e17783ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:31.778258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:31.780091 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:31.781433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 311/642] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:31.782829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.782829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.783109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.802464 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:31.803735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:31.805361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.894889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d6a2-0b3f-47f9-8db7-18a0c85f0edc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:31.908433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 312/643] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:31.909913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:31.957051 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:31.957597 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:31.958845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:31.959318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:31.959969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:31.973137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:31.974896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:32.003159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:32.004436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 332/644] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:32.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.061005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac03bf2-0eb2-4528-98b6-802daeb932df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.125474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.127527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.128014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.128509 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.140718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.143387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.151122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:32.159587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e067-d8cb-4d6d-aca9-2c8709d422e9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:32.160516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:32.162506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 313/645] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:32.164334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.202824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.203535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.203823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.204102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/2ca0990c5479452da837df05b9fbcacb/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.220594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=2ca0990c5479452da837df05b9fbcacb, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.222096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.236832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 333/646] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:32 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/2ca0990c5479452da837df05b9fbcacb/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:32.237702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.336505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:32.347228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e5d2-2e3f-4192-8a29-225f8ceedbec None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.379588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62ce4-0026-41c2-b1c0-3a47e4adfd89 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.398580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.399771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.400267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.400622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.402176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:32.402600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.405348 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.429080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.429949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.430237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.430511 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.430688 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 314/647] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:32.432083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.448218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:32.456261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:32.457275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:32.458571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 334/648] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:32.459274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.465351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:32.599864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e60fb-593a-4660-8029-3b00911c4eec None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.602594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d522f3-8b9d-4221-8fe8-b26ec2b1a4f0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.642733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.644104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.644842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.645329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.646283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:32.646964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.648990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.661234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.662302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.662819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.663321 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.664875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:32.665403 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.670078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.670242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 315/649] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:32.671376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.697738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 335/650] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:32.697738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.699580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:32.782467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:32.829180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfa5c3-fb4b-4580-a030-5caa55432359 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.836901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd83c9-64f2-4d30-81d4-0a1412d55761 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:32.881310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.881932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.882222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.882506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.883327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:32.883581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.884965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.887345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:32.888563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:32.889135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:32.889784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:32.891583 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=55a5560632ed452c8287a7c91b57e420) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:32.892767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:32.900377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 336/651] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:32.900587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:32.900672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:32.913910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 316/652] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/role_assignments?user.id=55a5560632ed452c8287a7c91b57e420&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:32.913910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.033221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.044419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa731f1-c53b-4b98-b4e9-ad9084db022f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.060574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd179f-3c3c-4c20-b86b-666e70a5314b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.081539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.096070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.096886 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.097221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.097606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.104241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.104916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.105206 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.105491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.107977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.110017 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:33.131608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:33.135379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 337/653] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:33.138138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.209103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.211523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.272037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561df40f-a44c-4e68-9277-b575d08a60f5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.326109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.326936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.328998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.330150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.330676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.347733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.349213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.366237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 317/654] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:33.371408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:33.372592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 338/655] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:33.373797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.377352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.420487 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.526234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75058a-8a81-47ad-903f-081997977d7e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.534423 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad85d0-138a-46d7-a699-b79516b9b2a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.584386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.585922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.586433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.586902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.587128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.588163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.588417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:33.588633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.589101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.589327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.591254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.612602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.614976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.625339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 339/656] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:33.625339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:33.640967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:33.642769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 318/657] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:33.645033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.690859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.767716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:33.775804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608d613-1032-4c2a-93a1-c75557eb2711 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.801606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad1b9-c587-43a0-990a-3b8476e42fb7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.817444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.818424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.818819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.819208 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.820057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:33.820476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.822211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.840436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 340/658] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:33.841520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:33.852924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:33.853982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:33.854523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:33.855119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:33.943598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:33.945363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:33.967364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31daa-4c46-4874-944f-af6e3ee9bb9e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:33.979449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.008210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.008999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.009387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.009706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.010531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:34.010915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.012612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.030976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 341/659] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:34.030976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.048895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.099634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 319/660] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:34.101142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.158011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91f229-7ce8-4560-8352-97f95c6ab5db None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.197501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.198290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.198579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.198863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.207540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.209136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:34.226027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:34.227046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:54:34.227990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:34.231972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 342/661] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:34.232960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.248653 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f1fb-7968-452d-92c8-85ff34317730 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.285168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.293764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.295031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.295522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.296139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.305193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.307266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.322761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:34.324683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:34.325568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:34.326755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:34.334106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 320/662] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:34.336371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.353843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13390e51-2eee-403b-9793-f2e7a3b39a03 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.398258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.398814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.399191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.399491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.416405 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.419195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:34.450124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:34.451431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 343/663] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:34.452615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.484359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6545-ef5d-47ab-956d-99edee2e5a70 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.550565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.551260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.551550 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.551825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.552701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:34.553016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.554539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852124d8-acf4-4a57-a7c4-05ce1f939498 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.575870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 321/664] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:34.576984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692aab-f370-4966-8429-1e5d48b5a777 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.577266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692aab-f370-4966-8429-1e5d48b5a777 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.579663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692aab-f370-4966-8429-1e5d48b5a777 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.583293 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 322/665] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:34.584279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692aab-f370-4966-8429-1e5d48b5a777 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.609377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e5d80-a890-4f8f-8e5f-dbb0002ab08b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.652535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.665381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:34.675168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.676562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.677094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.677596 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.679117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:34.679550 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.681796 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43848044-1e4a-4fc0-8b42-0ac01ac35403 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.715549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 344/666] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:34.715549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.715549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.715549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.749185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:34.751089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:34.772241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc692aab-f370-4966-8429-1e5d48b5a777 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:34.840491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 345/667] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:34.842261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:34.843786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:34.844334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:34.844946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:34.846628 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.847285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:34.847856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:34.850759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:34.877770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 323/668] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:34.877770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:34.996810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233245-3e1c-4f7e-bc00-3e00436b9fed None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.022875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.057657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8a54-be8b-4488-8bd6-653bc3afd124 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.058942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.060089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.060765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.061272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.076160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.112221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.112899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.113183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.113460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.114356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=b5a31865c8ed4ab78bfdb17780b61d41) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:35.114677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:35.116264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:35.131475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 324/669] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/role_assignments?user.id=b5a31865c8ed4ab78bfdb17780b61d41&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:35.132960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:35.172651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:35.177017 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:35.330181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146db0e-3b96-410f-bcf6-7f5ee4ccf1ae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.370544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 346/670] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:35.372788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:35.390729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.392131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.392578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.393028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.396258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.405856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:35.409069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b0dacce0-f61b-4ed0-8a63-605210d0d566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:35.436313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:35.437303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:35.438491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:35.441857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 325/671] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:35.443805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0dd1-1db6-45a5-9b8d-40e095c4dfbe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.444342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0dd1-1db6-45a5-9b8d-40e095c4dfbe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.445254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0dd1-1db6-45a5-9b8d-40e095c4dfbe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.446941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 326/672] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:35.447913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0dd1-1db6-45a5-9b8d-40e095c4dfbe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:35.490330 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.528717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b841-e7da-4723-948a-31a6342b0177 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.569662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.570510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.570797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.571078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.583794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:35.591974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:35.593813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 347/673] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:35.596170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:35.623005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0dd1-1db6-45a5-9b8d-40e095c4dfbe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.675373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.676660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.677136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.677606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.699319 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:35.701937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3e805efe-accc-4abf-9e4e-9d9a67893832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:35.733265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:35.735329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:35.737298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:35.737298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:35.737298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:35.737298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:35.737298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 327/674] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:35.737858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.738288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.738762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.761774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:35.763339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:35.771599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.822924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78d71-1762-40e1-ae73-a635d68275f1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:35.870543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 328/675] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:35.872516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:35.906359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:35.907583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:35.908402 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:35.908981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:35.910059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:35.912095 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:35.913100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:35.915821 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:35.939810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 348/676] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:35.941334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.039107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4fb2-7f07-4613-980c-e99b7e27b73c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.080513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.081328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.081604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.081907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.082660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:36.082977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.084727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.094261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 329/677] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:36.095144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.138494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4518e-c72a-4d64-8ae5-347a4c959aec None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.197536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.197863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.198863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.199152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.199501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.200504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:36.200820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.202317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.206559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.222434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 349/678] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:36.223412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.232188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2755-0106-4951-8047-a349addf9649 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.271092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.271934 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.272228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.272481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.284436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.285959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:36.304671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:36.306245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 330/679] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:36.308030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.367744 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338385-4a8f-4492-b498-44a33360288b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.406815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.407464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.407740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.407915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.408768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:36.409090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.410571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.425977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 350/680] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:36.425977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.464992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c91d4-5987-4642-83fd-7497c7f93eba None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.498474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.505578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.518038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.519234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.519690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.520126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.535725 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.537652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.552114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee65043-1c95-490e-b54a-87cf8736e491 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:36.559397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:36.560369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:36.561354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:36.561354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:36.561354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:36.561354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:36.565632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 331/681] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:36.566830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.592134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.593185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.593578 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.593986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.602703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.604367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:36.622014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:36.623263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:36.624367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:36.629701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 351/682] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:36.629701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.735599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43e-c576-45ec-8d30-28830bf957d8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.746985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55620111-f1b6-454e-977d-46e00f961cb1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:36.770279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.796098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.796776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:36.797074 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.797366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.797650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.798706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:36.798992 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:36.799196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:36.799292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.799668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:36.800689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.801047 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:36.801617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:36.804233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:36.813934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 352/683] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:36.817397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.830535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 332/684] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:36.832191 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:36.898098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:36.996215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fe437-d68a-446f-a244-335b5e6d323d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.007998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fbaf-75f9-4623-937d-1b8be3bcb135 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.069989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.072530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.073732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.074808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.077510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:37.078541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:37.082509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:37.083511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.084770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.085090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.085373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.097895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:37.099875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:37.112965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 353/685] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:37.114859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:37.123379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:37.125380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 333/686] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:37.127246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:37.234131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:37.320995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:37.324556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda230da-3b59-43b5-acf0-f63a2c4739f1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.347038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7249bb-aecf-4961-97f4-d06b677adbcb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.393780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.395219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.395970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.396739 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.398697 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=0cf7137aab9d4ebb8b83677cc4b0d817) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:37.399487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:37.403008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:37.404158 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.404947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.405241 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.405516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.406286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:37.408233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:37.417128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 334/687] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:37 2022] GET /identity/v3/role_assignments?user.id=0cf7137aab9d4ebb8b83677cc4b0d817&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:37.420595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:37.444791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 354/688] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:54:37 2022] POST /identity/v3/users => generated 254 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:37.575530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a7f5-c6f9-4e29-8a5a-87c5c4aa7cf6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.633727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.634436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.634728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.635015 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.635892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:37.636211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:37.637806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baa1ab-cf40-47b4-85a6-881ce0fbbe3d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:37.678267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 335/689] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:37.686814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:37.702728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbef4be-e502-4904-b800-017720f5473a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.703070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbef4be-e502-4904-b800-017720f5473a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.703296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbef4be-e502-4904-b800-017720f5473a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.705733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 336/690] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:37.717971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.718338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.718583 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:37.738040 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:37.739327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:37.769301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:37.855681 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 337/691] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:37.884404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:37.898631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:37.908735 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1adcd-8681-4dea-af10-4f51526ddc35 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:37.951053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:37.951981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:37.952363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:37.952738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.016663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931706a-be7f-4c6e-9c30-0b446eb7c9d0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:38.044911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.046578 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.078376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:38.079819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:38.080264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:38.080696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] PATH_INFO: `/v3/services/network` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.082123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.084179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:38.105012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:38.106513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:38.107643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:38.111574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 338/692] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:37 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:38.118448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:38.156231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:38.193697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:38.209740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 355/693] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:38.216859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:38.370724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0a78-f936-4650-a4a3-944ca5e0a185 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:38.411561 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cfede-e902-4f6a-b701-1081bec8688c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:38.484062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:38.485470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:38.486538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:38.487018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.500009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:38.501241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:38.501769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:38.502267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.503670 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:38.504242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.507009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.507788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.510808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.529849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 339/694] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:38.547670 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:38.555279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:38.557049 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:38.558819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:38.558819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:38.558819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:38.558819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:38.562234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 356/695] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:38.567283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:38.705479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:38.771279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:38.834760 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067167e-9bbe-4280-8a52-7396abf57cd3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:38.886105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e4a5d-5eb8-4836-b12c-9ddd22b5259b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:38.911800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:38.912497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:38.912802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:38.913089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.913809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:38.914126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.915528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.922621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 340/696] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:38.949779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:38.950471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:38.950757 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:38.951040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:38.951777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:38.952092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:38.953493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a3a3-4d72-4dde-9390-82014a9ccaab None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:38.968020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 357/697] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:38.976662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:39.128049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374626cd-0ada-4ec3-871c-b6f0b06433d4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:39.151320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f0f6-005b-4c86-8733-133d2e3917a8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.151766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f0f6-005b-4c86-8733-133d2e3917a8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.151990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f0f6-005b-4c86-8733-133d2e3917a8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.153652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 358/698] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:39.159448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.159860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.160108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.179383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:39.180756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:39.237148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:39.238530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.239033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.239524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.252594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:39.254201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:39.270404 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 359/699] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:39.280506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:39.283620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:39.286381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 341/700] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:39.291805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:39.297795 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:39.302010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:39.349349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:39.466338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a40b-54a8-4626-acbb-9eec8d85563e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:39.482848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f580e7-9223-4bf4-8400-12f37d752d95 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:39.563213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:39.564495 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.564776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.565031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.565872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:39.566183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:39.567651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:39.591564 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 360/701] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:39.594870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:39.595730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.596005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.596254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.602457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:39.623702 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:39.632229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:39.635838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:39.639043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 342/702] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:39.647237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:39.749582 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:39.754763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8637bc6-a55b-481c-8e3e-6cd1a236b345 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:39.794740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:39.795980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:39.796681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:39.797159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:39.798807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=2ca0990c5479452da837df05b9fbcacb) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:39.799338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:39.801543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:39.809472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 361/703] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:39 2022] GET /identity/v3/role_assignments?user.id=2ca0990c5479452da837df05b9fbcacb&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:39.828420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:39.940667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da489a-74b0-4ff7-bc0a-56576a138f2f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:40.000019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:40.001781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:40.002299 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:40.002789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:40.004220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:40.004782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:40.007476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:40.029717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 343/704] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:40.037543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:40.207967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2425-91e0-4dd5-b868-5519b222f0d5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:40.252692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:40.253981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:40.254494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:40.254994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:40.269281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:40.271191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:40.294267 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:40.295293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:40.296334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:40.296334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:40.296334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:40.296334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:40.298787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 362/705] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:40.307521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:40.510903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487577e-b13b-42d5-928d-6ba966bb2549 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:40.558738 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:40.580462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:40.581583 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:40.582111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:40.582612 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:40.584024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:40.584834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:40.587496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:40.611899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 344/706] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:40.622806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:40.646776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:40.750407 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936c9-db33-4cf7-ab8a-7abf1f4996db None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:40.797244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:40.798617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:40.799135 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:40.799633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:40.808905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:40.810767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bd136447-9fd3-4285-82b5-ffe29b002539 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:40.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:40.847576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:40.848850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:40.849456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 363/707] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:40.857953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:41.008214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638df9-bcd6-41fd-af91-a190a967b814 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:41.079458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:41.080559 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.081066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.081545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.083220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:41.083784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:41.086921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:41.133705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 345/708] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:41.137926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e05c-8489-4262-93ef-18d485bec266 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.138441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e05c-8489-4262-93ef-18d485bec266 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.138935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e05c-8489-4262-93ef-18d485bec266 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.141515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 364/709] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:41.143307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:41.169067 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.169630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.174356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.227070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:41.229118 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:41.244648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:41.412275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7bf35-f058-465d-9bbb-1a9cf44ce0c0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:41.436501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 365/710] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:41.495414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:41.496571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.497075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.497569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.531677 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:41.535910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:41.614723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:41.616162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dad3e-aab1-4b5d-919a-462f1c605c93 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:41.665192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:41.670011 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123d87-aa89-4d19-8bfe-a42be1ec0957 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:41.718098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:41.719281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.719575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.720438 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.721131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:41.722575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:41.743974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 346/711] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:41.746364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc3dd-5df3-4de1-9b46-2cedcdb4a7fe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.746931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc3dd-5df3-4de1-9b46-2cedcdb4a7fe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.747427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc3dd-5df3-4de1-9b46-2cedcdb4a7fe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:41.753181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:41.754254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:41.755114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 366/713] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 11 11:54:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:41.757728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 347/713] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:41.758649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:41.759691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.760202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.760693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.785037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:41.787551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:41.840351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:41.897489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf748-f698-4c04-95cf-400d6a440223 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:41.909544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 348/714] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:41.909544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:41.968757 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:41.970825 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:41.971614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:41.972202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:41.989459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:41.991664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:42.019696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:42.020676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:42.021588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:42.021588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:42.021588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:42.021588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:42.024650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 367/715] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:42.026245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.054871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:42.097425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ff13d-7fb0-47c3-ace1-e924774b4a9b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.153738 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.155042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.155556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.156012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.156931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:42.157476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.159088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.166078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 349/716] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:42.167240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.235610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819326a0-5897-4f09-ae1d-d955d126e377 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.259717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:42.285035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.286679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.287289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.287884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.305263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2d044-73c1-4293-b7c1-e170fecfc25f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.308831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:42.325636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:42.326679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:42.327531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:42.328826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 368/717] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:42.333073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.350610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.351393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.351682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.351963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.352744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:42.353263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.355896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.368916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:42.378549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 350/718] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:42.379452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.512956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb015e-3689-4b78-bb46-4f835090fb1e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.516806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cca8c7-70c6-440a-bb87-c2872a4c2446 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.566066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.567065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.567674 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.568129 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.569121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:42.569617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.571233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.578651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.580036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.580193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 369/719] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:42.581066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.581432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.582207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.583769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:42.584849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.587920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.603185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 351/720] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:42.624598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.665032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:42.750316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406af54-01ef-4f7f-a305-64588bf0633c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.756386 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:42.815902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.817545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.818084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.818516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.820327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245c9f5-9971-49d5-b9ad-0010b9b9507e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:42.832884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.835068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-04aeda04-b0fc-4069-8e7e-a319fd43b6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:42.864709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:42.866336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:42.868366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 370/721] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:42.869380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99d3d-c436-400e-8d9e-7662a4ca8f05 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.869380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99d3d-c436-400e-8d9e-7662a4ca8f05 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.869380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99d3d-c436-400e-8d9e-7662a4ca8f05 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.871190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 371/722] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:42.872087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b99d3d-c436-400e-8d9e-7662a4ca8f05 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:42.890758 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:42.892134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.892724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.893356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.912531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:42.917491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-699e0fd6-844b-499c-bce0-e4a8dfd6ac47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:42.942801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:42.944129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:42.945188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:42.945188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:42.945188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:42.945188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:42.947750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 352/723] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:42.949625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:42.950270 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:42.950782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:42.973597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:42.974967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:43.038114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b99d3d-c436-400e-8d9e-7662a4ca8f05 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.041233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.063457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 353/724] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:43.064657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.114512 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.131906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.133511 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.134062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.134549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.135560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:43.136733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.138745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:43.161715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 372/725] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:43.161715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.214208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82907bc-d175-417a-bc2a-8f72811d958b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.275838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.277063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.277492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.278053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.279450 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:43.280017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.283231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:43.306783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 354/726] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:43.307777 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.332509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2c08-d0d9-4929-8647-6c09f990c496 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.380395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.381051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.381344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.381790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.382560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.384482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:43.405292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 373/727] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:54:43 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:43.406577 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.440321 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a88128-8a74-48c0-b67a-59331cac91a9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.442739 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.471826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.511104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.512009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.512558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.513119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.514488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:43.515019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.518324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952a78-eaed-40a5-a6b5-a9ece0ba19b6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:43.527161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 355/728] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:43 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:43.528950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65739449-ae1f-463b-893d-b29337af02cb None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.529326 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65739449-ae1f-463b-893d-b29337af02cb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.529740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65739449-ae1f-463b-893d-b29337af02cb None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.531629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 356/729] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:43.547806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.548341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.548894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.587458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:43.588922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:43.671834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe7ba-65cc-42df-a7a5-ee5e8368685e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.728293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.729259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.729615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.729994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.733013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 357/730] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:43 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:43.740986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.744282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:43.766597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:43.773144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:43.775683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:43.777566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 374/731] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:54:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:43.781786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:43.886866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.911006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9e737-90f5-4844-a8a4-62ffa2fc25a6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.933085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:43.950366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633984f-531e-4f55-ae81-c1aa856d0ed2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:43.969116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:43.971005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:43.971786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:43.972266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:43.974113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:43.974915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:43.977781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:44.003990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:44.005880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:44.006224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:44.006518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:44.012812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 375/732] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:54:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:44.017942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:44.023313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:44.032155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:44.033157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:44.034035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:44.038041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 358/733] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:44.044066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:44.210236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dc206-2b56-4c35-8b8a-275b70203c1b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:44.254613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:44.255279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:44.255565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:44.255841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:44.259722 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0536c2-a3a9-4f9a-a179-5c854c60c5ab None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:44.288255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:44.342907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:44.343602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:44.343890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:44.344173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:44.344996 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:44.345312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:44.346779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:44.368488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 359/734] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:44.374497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:44.404570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:44.408998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:44.410538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:44.607801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d0900-6fcf-4e7f-a46e-a2a6010cae2b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:44.645368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 376/735] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:54:44.652930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:44.706627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:44.707486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:44.707778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:44.708053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:44.735166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:44.737263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:44.743671 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:44.779625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:44.780603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:44.781497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:44.781497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:44.781497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:44.781497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:44.790537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 360/736] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:44.810382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:44.858547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32920c72-6960-40b0-9278-8e3befbbe67a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:44.911282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:44.912212 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:44.912502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:44.912782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:44.928005 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:44.929629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:44.940983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:44.961161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:44.963454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:44.965616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:44.965616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:44.965616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:44.965616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:44.965616 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 377/737] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:44.973807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.010011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345452b-f65e-4358-8f60-dfbf74c87c0d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.081967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.083100 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.083598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.084094 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.085754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:45.086455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.089042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.113035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 361/738] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:45.119836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.207552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:45.227034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7a52-13ac-405c-82da-58f7c200a432 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.299672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.300635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.301091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.301523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.302721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:45.303166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d84dc-5ec8-412f-aa11-b869f46e81eb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.303276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.305865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a942-fc35-4719-bab4-a2e8fc7a5fd2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.330052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 378/739] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:45.332586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71fcfc-51ce-4535-a9ff-f26ecc3f32f1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.333111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71fcfc-51ce-4535-a9ff-f26ecc3f32f1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.333218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:45.334087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71fcfc-51ce-4535-a9ff-f26ecc3f32f1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.339665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 379/740] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:45.341186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71fcfc-51ce-4535-a9ff-f26ecc3f32f1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.349247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.350543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.351054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.351540 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.360123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.362017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-236d1716-6672-45ba-9fca-987df5468278 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:45.378352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:45.379361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:45.380309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:45.383062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 362/741] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:45.384592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.385178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.385699 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.405981 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:45.408491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:45.509695 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 363/742] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:45 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:54:45.509695 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.519317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71fcfc-51ce-4535-a9ff-f26ecc3f32f1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.568027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.569187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.569654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.570152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.577863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.580255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:45.596535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:45.597491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:45.598435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:45.601088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 380/743] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:45.602163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.624657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:45.691618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893b434-6017-4dee-822f-0c308d5830f2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.726134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c98b5-a4f6-46c3-9e83-bac1a76be6be None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:45.726375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:45.774359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.775073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.779347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.779347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.779347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:45.779347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.780008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:45.780008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:45.780308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:45.780522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:45.781732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:45.784555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.785495 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:45.818930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 381/744] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:54:45 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:45.823319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:45.835771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 364/745] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:45.843686 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:46.065125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957e81-af45-4ce6-9e36-beb71f2c5f79 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:46.090986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:46.164850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:46.166810 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.167690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.168768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.204246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65811ced-7a79-4952-94da-c9f95ea05316 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:46.234287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:46.278178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:46.279792 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:46.285399 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:46.286467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.286888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.287300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.288623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:46.289188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:46.291971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddb9df-2959-4685-80b7-86fb32ed7a16 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:46.314077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 382/746] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:46.315928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f787e-aa3e-4e56-b075-83cc92c9de83 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.316474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f787e-aa3e-4e56-b075-83cc92c9de83 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.316970 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f787e-aa3e-4e56-b075-83cc92c9de83 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.318895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 383/747] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:46.320360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83832-167c-4a40-8bab-2b55849e912a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.320891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83832-167c-4a40-8bab-2b55849e912a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.321373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83832-167c-4a40-8bab-2b55849e912a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.323197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 384/748] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:46.324605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5448-8b74-4d6f-b241-2948c75338dd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.325109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5448-8b74-4d6f-b241-2948c75338dd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.325601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5448-8b74-4d6f-b241-2948c75338dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.343319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-819d5448-8b74-4d6f-b241-2948c75338dd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:46.344841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-819d5448-8b74-4d6f-b241-2948c75338dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:46.430645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 365/749] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:54:46.430645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:46.432215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 385/750] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:46.433515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31f33-e424-4416-a72b-05024849b99c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.433833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31f33-e424-4416-a72b-05024849b99c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.434116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31f33-e424-4416-a72b-05024849b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.469223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eab31f33-e424-4416-a72b-05024849b99c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:46.471361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eab31f33-e424-4416-a72b-05024849b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:46.552068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:46.552454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:46.581802 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 386/751] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:46.581802 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab31f33-e424-4416-a72b-05024849b99c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:46.613831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e171a-cb54-4653-93a1-359983c5fab6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:46.675043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:46.676155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.676655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.677155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.678813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=9c42d851cdef43fcb152473bebeb08f5) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:46.679332 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:46.681882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:46.705696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 366/752] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:46 2022] GET /identity/v3/role_assignments?user.id=9c42d851cdef43fcb152473bebeb08f5&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:46.705696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:46.773872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab31f33-e424-4416-a72b-05024849b99c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:46.843890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:46.846529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.847223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.847878 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.873530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:46.876233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:46.911199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:46.912260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:46.913296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:46.913296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:46.913296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:46.913296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:46.914025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 387/753] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:46.915504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:46.929626 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654193-cb0b-4d63-a4a6-3d3c9ce25a69 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:46.993275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:46.994682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:46.995207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:46.995715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] PATH_INFO: `/v3/services/network` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:46.997220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.000237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.004071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.010040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:47.025034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:47.026389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:47.027448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:47.030056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 367/754] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:46 2022] GET /identity/v3/services/network => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:47.031144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.113555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2489-ca0b-4443-ae87-fc9eee291f10 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.182712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.184070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.184568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.185047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.192132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064c7c8-f566-4352-9d37-2ff357720743 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.205492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:47.221507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:47.222572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:47.223413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:47.224551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 388/755] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:47.225865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.237947 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.239009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.239508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.240039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.240992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:47.241519 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.243952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.261525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 368/756] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:47.262777 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.348099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.403814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e1d95-2618-4f01-95b4-feb718a7211f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.414054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.420377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643dfb-dba0-4a4e-bf18-49848205275d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.452570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.454916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.455429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.456055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.457098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] RBAC: Adding query filter params (name=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:47.457620 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.459837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.467263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 389/757] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:47.469200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.499898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.501878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.502595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.503093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.504540 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:47.505189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.508133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.526360 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 369/758] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:47.528028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.625340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667d3e-0e0a-4fed-bb4c-1621cd36a9ba None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.678232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.679420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.679937 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.680436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.689332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.691195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:47.706707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:47.707702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:47.708623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:47.711579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 390/759] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:47.712835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.747271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ca0dc-ba71-42db-9a11-18030dd83419 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.753516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.801763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:47.831333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.832477 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.832993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.833472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.835133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:47.835713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.838629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.847613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebb273-e780-4f96-b8ff-4028a47ce6a9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:47.853908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 370/760] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:47.855591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:47.889987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:47.891182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:47.891748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:47.892231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:47.904782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:47.906691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:47.927317 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:47.928354 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:47.929274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:47.929274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:47.929274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:47.929274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:47.931659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 391/761] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:47.932762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.008263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e21242-0e67-48dc-a94b-a3337f08effc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.022946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:48.043640 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778c5cd-63fe-4963-8dfa-86d09b3032e1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.049481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.050424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.050889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.051334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.052399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:48.052937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.054880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.069596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 371/762] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.070937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.081904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.082586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.082874 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.083288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.084754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] RBAC: Adding query filter params (interface=public, service_id=012b26abf95b4b90b2fe77d9f413d944, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:48.085093 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.086546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.100832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 392/763] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:54:47 2022] GET /identity/v3/endpoints?service_id=012b26abf95b4b90b2fe77d9f413d944&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.100832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.132608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:48.197839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669cc75-5c0c-43de-bb6c-a5126ff85901 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.222509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7b65-5eca-4f5d-8993-32743b7b5ea2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.240633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.242170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.242671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.243171 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.244124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:48.245267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.247236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.268045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 372/764] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.268045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.270803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.271953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.272448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.272931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.274770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=2ca0990c5479452da837df05b9fbcacb) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:48.275289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.278007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.302188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 393/765] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/role_assignments?user.id=2ca0990c5479452da837df05b9fbcacb&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.303931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.345468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:48.422005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00423b-36ab-423e-a886-a62ed59629e5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.424870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:48.470994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.471618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.471906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.472192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.472919 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.474392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.490353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 373/766] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.561278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72ff79-775b-4761-98f6-24cebe6f7791 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.613227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.614124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.614416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.614699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.624454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.626061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-31997e73-e76d-4d37-ac9d-5068f89903f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:48.643084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:48.644113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:48.645178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 394/767] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:48.652302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.750419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9587-d766-47b3-8b1a-69c0a52359d7 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.751463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9587-d766-47b3-8b1a-69c0a52359d7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.751973 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9587-d766-47b3-8b1a-69c0a52359d7 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.761911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 395/768] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:54:48.767050 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.767553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.768053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.778257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27db75-77df-417c-b6e9-764e0c054199 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:48.803818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:48.806068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:48.823633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:48.824368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:48.824663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:48.824951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:48.825876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:48.826199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:48.828237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:48.853196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 374/769] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:48.859140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:48.914211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:48.982527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc9418-7755-4db6-8bd5-ce723e929b7d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.012274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.024573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 396/770] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:54:49.031203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.032393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.032872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.033337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.045168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.047004 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:49.067487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:49.068471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:49.069420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:49.069420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:49.069420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:49.069420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.071784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 375/771] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:49.077901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.103163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.210917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.225698 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861a62c-4696-4a83-b357-a96f37c200dd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.236099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1db64-e87f-4802-8d00-b8dc9d07175a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.268493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.269606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.270072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.270526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.271423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:49.271891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.273476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.274860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.275684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.275963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.276237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.276902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.278528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7d7de759-9e46-499d-b34d-4e0071926c6e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:49.293336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:49.294472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.295342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 376/773] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:49.296207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb830-2cd1-4261-9e0a-8a16eff510b8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.296207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb830-2cd1-4261-9e0a-8a16eff510b8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.296207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb830-2cd1-4261-9e0a-8a16eff510b8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.296939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 397/773] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:49.297120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 377/774] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:49.302395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb830-2cd1-4261-9e0a-8a16eff510b8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.304239 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.400784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.457513 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df9785-6e32-4597-ba7d-2682a611b3de None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.461121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb830-2cd1-4261-9e0a-8a16eff510b8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.510768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.511753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.512167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.512579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.513804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:49.514264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.516801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f970e-c348-49da-9062-28b219f339f6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.517439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.518749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.519278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.519971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.529721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.530096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 398/775] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:49.530654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.533927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.534639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.538531 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:49.551499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:49.553329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:49.555200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.556257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 378/776] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:49.556992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.560261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.574287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:49.576856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:49.702281 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba784-5bfc-4373-9c6d-650d1793c379 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.717812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 399/777] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:49.720249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.764232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.765110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.765497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.765937 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.766813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:49.767219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:49.768716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:49.781804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 379/778] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:49.781804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:49.841380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.864276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:49.909264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6136-1c17-42a1-b729-48774fa13502 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.934472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b42abe-490c-423a-a313-d64265ab5496 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:49.951760 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:49.952572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:49.952861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:49.953112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:49.967269 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:49.976698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:49.978562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:49.980748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:49.982655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 380/779] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:49.982892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.011025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.012771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.013862 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.014662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.017191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:50.018125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.021704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.047052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 400/780] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:50.049232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.107570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8c5fe-3704-409a-acc1-218a1f533594 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.154258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:50.168168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.169229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.169768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.170232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.171244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] RBAC: Adding query filter params (interface=public, service_id=4a88cb34c2f947c3be5c5d974d896295, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:50.171751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.173461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.183383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 381/781] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:54:49 2022] GET /identity/v3/endpoints?service_id=4a88cb34c2f947c3be5c5d974d896295&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:50.184522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.200706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6fc8-faba-43d4-a964-2dce385b2cbb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.218876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:50.247673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.249270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.249856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.250362 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.251967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:50.252523 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.254335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.263906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 401/782] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:50.270234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.334392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1376a3-0aac-4535-9e65-c110be5ac243 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.374591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.376548 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.377043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.377534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.380189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=80890a6017284bf090892fbcbb618f7f, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.381928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.402787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 382/783] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:50 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:50.403934 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba59e-f344-4c20-b93b-0d706a7028ec None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.404301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.477304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.478553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.479114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.479634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.480572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.482318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8392153-dd0a-4cd1-bcc2-b41a8bb6c8f0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.489894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 402/784] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:50.551329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:50.626340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb10cd-02bb-4453-b12f-0eb3e806444a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.657045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb88be0-6cd8-4f49-8b6e-fcf0f55dae41 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.657660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb88be0-6cd8-4f49-8b6e-fcf0f55dae41 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.658096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb88be0-6cd8-4f49-8b6e-fcf0f55dae41 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.662260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 403/785] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:54:50.669107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.669824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.670323 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.677822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.678768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.679053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.679328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.694126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:50.695973 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:50.699123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.700726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:50.728984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:50.730009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:50.730907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:50.730907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:50.730907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:50.730907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:50.730907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 383/786] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:50.737121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.783470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:50.829038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 404/787] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:54:50.853031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.878659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7331b-8579-4eb3-939c-7c05bf0be05d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:50.882217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:50.919864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:50.920563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:50.920856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:50.922136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:50.923535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:50.923855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:50.925279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:50.939932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 384/788] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:50.947433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:50.975623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5d652-8d16-4ef7-89aa-2b68c2d0f624 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.015348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.016196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.016529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.016808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.029335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:51.039201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:51.041468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:51.043367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 405/789] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:51.053828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.085532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb455fea-12f6-4d1c-b0aa-50db6447b8a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.122859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:51.203942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.205861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.206717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.207477 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.217705 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e46ee-02b7-4796-a984-a332fddeae39 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.243187 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.246392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.263230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.264029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.264179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.264533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.265230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:51.269719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:51.274565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:51.276313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:51.280027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:51.284083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.288152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 385/790] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:54:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:51.289133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.289845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 406/791] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:51.294986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.350192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:51.474194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:51.485061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ebf7-b31c-474e-a606-1d1f716543ae None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.524445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4730-80fa-4ab2-9bc7-8a653ad060ea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.526013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.527027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.527323 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.527608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.543276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.545242 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:51.579828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:51.583302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:51.589393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:51.589393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:51.589393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:51.589393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:51.589393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 407/792] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:51.593825 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.602849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.603544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.603830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.604083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.604947 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:51.605259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.611979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.644176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 386/793] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:51.654882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.755741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:51.781837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514fff-c4ed-4950-b546-2e718bb29378 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.833969 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.835109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.835395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.835672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.848001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea595-bff3-46ec-965e-8567f4888208 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:51.849533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.851363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9b8332ba-6979-457b-9caf-84b425f3bb9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:51.872761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:51.873790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:51.874718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:51.874718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:51.874718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:51.874718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:51.874718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 387/794] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:51.875856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ae65-50a6-49b2-a5d7-ce97091922e1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.876226 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ae65-50a6-49b2-a5d7-ce97091922e1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.876514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ae65-50a6-49b2-a5d7-ce97091922e1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.882159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 388/795] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:51.882355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ae65-50a6-49b2-a5d7-ce97091922e1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:51.899692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:51.900814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.901316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.901944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.902931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:51.903467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:51.905150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775bb5f-b1d9-4026-a7d0-a6f3a21cd487 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:51.925808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 408/796] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:51.926224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:51.926850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:51.926850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:51.937994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:51.953816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:51.955570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:52.011345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ae65-50a6-49b2-a5d7-ce97091922e1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.050009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 409/797] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:52.050631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.050808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.059925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.060656 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.060991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.061279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.062119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:52.064137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.066050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.085640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 389/798] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:52.086382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.193457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d211-9c70-4eaa-a560-4aec9d8d193f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.206044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35c2d0-a423-464c-933a-4f987b9cc93b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.237424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.238651 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.239231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.239808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.242619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.246214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.247159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.247524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.247873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] PATH_INFO: `/v3/services/network` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.248838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.251153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.252146 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.254031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:52.264946 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:52.266076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:54:52.267503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:52.271857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 390/799] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:52.272905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:52.275723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:52.277061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:52.278508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:52.286091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 410/800] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:52.287558 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.341555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.404416 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad7dfa-6618-464b-bbdf-6814c7cac3ea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.409891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195b30-2c33-4449-9ff8-0aeaf7ec9d3d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.446791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.447614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.447898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.448171 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.458791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.460534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.460786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.461872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.462374 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.462917 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.463957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] RBAC: Adding query filter params (name=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:52.464495 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.466180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.472079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 411/801] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:52.473196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:52.481041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:52.482034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:52.482932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 391/802] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:52.483886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.551507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.606312 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f782-3225-492e-bb33-01032d2d9a2e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.610151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2de2b-a310-4072-9723-8c2e3043e4ed None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.629916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.650813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.651499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.651640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.651833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.652065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.652821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:52.653137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.653474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.654037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.654528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.654636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.656170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:52.656886 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.658798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.666224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 392/803] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:52.667403 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.686439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 412/804] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:52.687917 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.787078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ac236-8159-40ce-a10e-f2ea416fcf9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.805609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36dddf-ec4f-4b20-9799-85d4b10cea16 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:52.824615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:52.830428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.831988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.832347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.832707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.833590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:52.834064 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.835974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.848429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:52.849480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:52.849975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:52.850498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:52.850751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 393/805] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:52.851629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.852139 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:52.853984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:52.881041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 413/806] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:54:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:54:52.889892 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:52.930917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:53.055768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266b60f-dc43-4bdc-9b5a-aa5de23f192e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.118629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.119493 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.120185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.120185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.135856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:53.137378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:53.157079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:53.158087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:53.158986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:53.158986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:53.158986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:53.158986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:53.158986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 394/807] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:53.165868 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236517f-dee9-4f94-952b-a0a71e807dad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.169857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:53.253928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.254846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.255236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.255520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/9c42d851cdef43fcb152473bebeb08f5/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.273927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=9c42d851cdef43fcb152473bebeb08f5, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:53.275716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:53.284505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:53.294155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 414/808] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:52 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/9c42d851cdef43fcb152473bebeb08f5/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:53.304372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dfebc-69e3-47f4-95c8-128c4bec9028 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.348354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.349012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.349304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.349585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.350404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:53.350723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:53.352229 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8aed7-799d-42e9-a533-7aa0d581da34 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:53.374308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 395/809] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:53.379697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:53.426322 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:53.431256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f270-f5b2-40e1-92a7-82bb97d8a9ed None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.431858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f270-f5b2-40e1-92a7-82bb97d8a9ed None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.432223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f270-f5b2-40e1-92a7-82bb97d8a9ed None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.435382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 396/810] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:53.444577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.445633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.446173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.483764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:53.485970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:53.549969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e29a-3a71-417a-a7ba-ebd219ea82ce None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.588952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.589789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.590082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.590363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.598565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:53.600300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:53.615025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:53.617000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:53.619108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 415/811] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:53.627794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:53.645432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 397/812] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:53.669578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:53.781207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efdea5-7fd6-43fa-83d0-d21b6e2fce89 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.792791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:53.808182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:53.827121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.828307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.829351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.830267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.845869 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:53.850502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf8ee-f68b-4ad9-9ca9-3e93de9e797d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:53.855019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:53.856016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:53.856976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:53.860134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 398/813] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:53.866184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:53.928455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:53.929629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:53.930203 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:53.930693 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:53.932986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:53.933575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:53.937908 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:53.960562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 416/814] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:53.969706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.018681 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc9a0a-b7c0-4de7-a70d-12aa9bebe9b3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.073189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce7fc-79c3-44c4-a891-09cd135da72e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.085209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.086421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.086921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.087570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.089122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:54.089711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.092645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96501c-93d1-4ed9-9c4e-f68b89655f1b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.103114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.112970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 399/815] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:54.114283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b1ff-0460-4dca-9c64-134719477c8a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.114510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.114644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b1ff-0460-4dca-9c64-134719477c8a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.114840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b1ff-0460-4dca-9c64-134719477c8a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.117789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.117956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 400/816] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:54.118039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.118039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/2ca0990c5479452da837df05b9fbcacb/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.118172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1b1ff-0460-4dca-9c64-134719477c8a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.122772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=2ca0990c5479452da837df05b9fbcacb, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.126202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5949e-9d85-4d22-9024-701bdcfe2fc4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.144313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 417/817] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:54:53 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/2ca0990c5479452da837df05b9fbcacb/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:54:54.145944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.146282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.146557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.151933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.180107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:54.184658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:54.291846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 418/818] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:54.317623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.327829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1b1ff-0460-4dca-9c64-134719477c8a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.381523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.382527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.383337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.383634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.408176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.410438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:54.440541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:54.443535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:54.446102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:54.446102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:54.446102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:54.446102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:54.446102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 401/819] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:54.447183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab2d2e-ee61-45c3-bc48-65063742ba35 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.450413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.484519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.490761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.491630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.491899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.492192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.492902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.495251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:54.532392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:54.534350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:54.537395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 419/820] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:54.540867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.542953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.649372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0861b7-0374-420f-97c7-a491e70b9386 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.653815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7f47-2dcf-4bd7-a8c4-749dc34bf840 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.690375 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.691122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.691410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.691729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.692445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:54.692750 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.694185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.699439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 420/821] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:54.704667 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.724138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.725786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.726683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.727467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.729130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:54.730312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.733413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.758708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 402/822] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:54.765041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.789865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.854210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9ed18-cc08-4932-a872-c7d5e179a6fd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.893051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.894145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.894720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.895202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.896121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:54.896632 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.898410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:54.904792 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 421/823] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:54.910906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:54.934205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6d89-1ac1-44a4-83ee-6cd3950facd9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:54.949134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:54.975940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:54.977199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:54.977718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:54.978203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:54.991079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:54.992978 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:55.016355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:55.017378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:54:55.018387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:55.023556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 403/824] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:54:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:55.024729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:55.098596 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24f05a-a698-47c2-95a9-4bb15785a8c4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:55.132704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:55.147961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:55.149169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.149604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.150090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:55.151006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:55.152724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:55.181585 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 422/825] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:54:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:54:55.226782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27fc5-5520-4f51-a8a4-94696fb89cff None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:55.311546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:55.312742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.313295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.313803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:55.315726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:55.316309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:55.319265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7b7d3-8101-413f-9d46-c4d7e890ff37 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:55.348328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 404/826] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:54:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:55.353918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:55.474918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b5fd1-7210-43b3-9e48-ab09e67795c9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:55.512551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:55.513550 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.513875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.514156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:55.572709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:55.590875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:55.592568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:55.644963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b3f9-7148-4af5-8262-8099e8a7f8e6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.645511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b3f9-7148-4af5-8262-8099e8a7f8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.645511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4b3f9-7148-4af5-8262-8099e8a7f8e6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:55.647469 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 405/827] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:55.653076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.653374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.653657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:55.675885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:55.677109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:55.712184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:55.718271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 423/828] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:54:55.724273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:55.792009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 406/829] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:55.816811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:55.865619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:55.907660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316a20-5f10-41f2-a61e-7d4b29fe76ec None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:55.977777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a111b-a857-4719-974d-a62b922003da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:55.979127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a111b-a857-4719-974d-a62b922003da None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:55.979617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a111b-a857-4719-974d-a62b922003da None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:55.980102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a111b-a857-4719-974d-a62b922003da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.000927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a111b-a857-4719-974d-a62b922003da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.003338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a111b-a857-4719-974d-a62b922003da None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c31a111b-a857-4719-974d-a62b922003da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:56.038887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:56.040429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:56.041629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:56.041629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:56.041629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:56.041629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:56.048691 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 424/830] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:56.050340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a111b-a857-4719-974d-a62b922003da None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.152762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe90852-843c-4df5-ae17-c90f1071b84d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.193105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.194469 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.195019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.195495 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.208346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:56.216888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:56.217895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:54:56.218740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:56.226192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 407/831] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:54:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:56.227434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.239751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a111b-a857-4719-974d-a62b922003da None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.248397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:56.300149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:56.311078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.312351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.312846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.313936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.315473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:56.316038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.318912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.350220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 425/832] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:56.352953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.355355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95876-f536-40e1-88c4-7b48c0560166 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.397451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.398717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.399213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.399690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.400665 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:56.401212 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.403029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.418162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 408/833] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:56.419533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.505510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a338a6-573e-4af5-84b3-793b4ba8c09a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.543631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:56.562018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.563753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.564257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.564744 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.572234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc016228-6df5-4ac3-a1fa-3541ddd1b3fe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.577344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.580947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-94c99cc2-2a1c-4512-a7e7-c4437e455e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:56.598081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:56.599741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 426/834] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92955ff-c40b-4350-a191-ad60ff72f59c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.601108 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92955ff-c40b-4350-a191-ad60ff72f59c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.601891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92955ff-c40b-4350-a191-ad60ff72f59c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.603254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 427/835] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:56.608613 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92955ff-c40b-4350-a191-ad60ff72f59c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.630209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.631714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.632227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.632726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.651848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.654728 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.680808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b7d754eb-8c72-49a0-aeb9-3d8e6e7e68da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:56.691129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:56.692120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:56.693067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:56.693067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:56.693067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:56.693067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:56.693825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 409/836] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:56.695773 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.696231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.696688 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.722764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:56.724587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:56.780717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92955ff-c40b-4350-a191-ad60ff72f59c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:56.843215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 410/837] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:56.844571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.857996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:56.859349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:56.860401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:56.861183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:56.863045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:56.863869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:56.867391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:56.894305 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 428/838] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:56.896161 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:56.930169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:56.981804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b492e3-bbdd-4bb3-939c-04c50fcddc21 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.025751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.026714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.027017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.027269 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.028069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:57.028379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.029770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.049726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 411/839] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:57.049726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.103985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd41cb-8b66-42d1-b549-2ebf0edc23de None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.133585 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:57.171774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.173614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.174110 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.174570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.192658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:54:57.205794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:57.207623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 429/840] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:57.211320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.231866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b902b2c-14e3-46f5-8982-aaf8c2be7547 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.270561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:57.296617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.298292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.298768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.299230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.300580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:57.301105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.303370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.313164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 412/841] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:57.314710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.410995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe0e2-01c9-47a8-9f39-a9d1981dff93 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.489880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.491837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.492684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.493443 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.510736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.515173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.553694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaadb88-29c7-4243-8c53-d9e8918fbd5e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:57.553994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:57.555771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:54:57.557652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 430/842] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:57.564629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.617484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.618588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.619145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.619428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.620352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:57.620671 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.621368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:57.625275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.645133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 413/843] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:57.653736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.712344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec8b9-d27b-4602-91c4-8403c94316a1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.737415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:57.755471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.756168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.756454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.756746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.757845 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:57.758166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.760105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.798738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 431/844] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:57.803007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:57.819203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c9178-9020-489c-81b2-d05103d7a97e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:57.865407 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:57.866267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:57.866654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:57.866856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:57.878925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:57.880401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:57.904219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:57.905789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:57.908556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:57.908556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:57.908556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:57.908556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:57.911001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 414/845] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:57.912041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.000586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.022430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdafeb-82b3-4f79-80e2-9cf726b926ae None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.046183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.053961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3415f7-0bba-4601-88b7-faf73769c63d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.075803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.077081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.077504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.077971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.102943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.104288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.104770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.105245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.106604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:58.107150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.109695 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59ce7a-b0fc-4374-881c-62fa74c04c63 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.134710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 415/846] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:54:58.134710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4d98-9c48-4d71-af93-a514542eb565 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.134710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4d98-9c48-4d71-af93-a514542eb565 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.134710 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4d98-9c48-4d71-af93-a514542eb565 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.140659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 416/847] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:58.142386 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb4d98-9c48-4d71-af93-a514542eb565 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.188134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.189773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487896-9d67-40c3-884a-664ab29ba63b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.299245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.315121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 432/848] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:54:58.319661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb4d98-9c48-4d71-af93-a514542eb565 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.324485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.324985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.325483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.355350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:58.357357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:58.361243 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.362098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.362410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.362690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.370955 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.372424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:54:58.388329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:58.390632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:54:58.394011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:58.396157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 417/849] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:58.396157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.418547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.493971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 433/850] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:54:58.493971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.555101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49915ef3-cfbe-480d-8dc6-dd27f9b64f82 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.618528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010e7a7-b7dd-448b-86c0-ff4e05b50ffd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.624822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.626863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.627449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.628059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.654314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.657361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.675657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.686284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.686996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.687281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.687562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.689154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:58.689549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.691098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:58.701137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:58.702125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:58.703048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:58.703048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:58.703048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:58.703048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:58.711634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 418/851] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:58.713078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.713726 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 434/852] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:58.718076 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.843408 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:58.853105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47007c1-3172-4d5f-874b-ca22756b58f6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.898544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20e811-201d-4350-aea1-db00ad48fe1d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:58.900869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.901634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.909725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.909725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.909725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:58.909725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.909725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.922103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 435/853] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:58.925720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:58.945443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:58.947308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:58.947788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:58.948065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:58.948743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:58.950518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:58.963790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:58.964853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:54:58.965730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:58.970396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 419/854] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:58.971353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.009010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.129554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f826572-aae1-4fa3-9e29-8f89bdfe0642 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.162371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5047acd-9da2-421d-a13f-aaad71e772cc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.197684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.198851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.199338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.199847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.204553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.206263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.206556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.206836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.209183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.216615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.218128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:54:59.234922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:59.236610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 420/855] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:59.237605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.324196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.326971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.386489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bf97-a19c-48ba-865a-82c49d9e5439 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.391880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.436057 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.436724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.437009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.437297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.438104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:59.438416 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.439812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.445426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 421/856] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:59.446220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.487370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 436/857] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:54:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:54:59.488602 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.547956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.600586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6851-11e1-41bd-89c9-47b376466446 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.626810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7011c3b-62fe-4d5d-bbae-e0266db96942 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.639286 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.639879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.640170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.640447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.641835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:59.642155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.643609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.657316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 422/858] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:59.658307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.679117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.679814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.680103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.680382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.681246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:59.681566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.683043 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.697731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 437/859] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:59.697731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:54:59.702164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.867471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024216a3-49d8-4d96-8778-3e7ae429f778 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.878229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9bf5-d83e-4dce-a3c3-675eab74e433 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:54:59.889833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:54:59.915503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.916350 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.916635 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.916915 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.930586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:54:59.931390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.931565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.931699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.931982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.932858 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=9c42d851cdef43fcb152473bebeb08f5) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:54:59.933035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.933167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:54:59.934701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9046c-e2d5-418e-a90e-7240d80d54c3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:54:59.942260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 438/860] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/role_assignments?user.id=9c42d851cdef43fcb152473bebeb08f5&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:54:59.943684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897cfb-2127-4483-9573-4e4f60eb1dd0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.943974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897cfb-2127-4483-9573-4e4f60eb1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.944258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897cfb-2127-4483-9573-4e4f60eb1dd0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.949884 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 439/861] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:54:59.951899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:54:59.952539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:54:59.952842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:54:59.965114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:54:59.966238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:54:59.967165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:54:59.967165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:54:59.967165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:54:59.967165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:54:59.969342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 423/862] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:54:59.977585 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:54:59.977585 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:54:59.979035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.077719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:00.090148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 440/863] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:54:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:00.113471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.173866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581503-575d-49e1-802f-a7d60ccc08d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.220744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.221432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.221737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.222018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.222751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:00.223062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.224450 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.241761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 424/864] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:00.246127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.283721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2149c72-8372-43c1-96c2-c710bae39d88 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.289940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:00.328151 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.329543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.329861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.330137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.330801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.332224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:00.345572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:00.347488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:00.349179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 441/865] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:00.356663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.386467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5bb5-65f2-4bbe-b8b5-5559f6b8fe01 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.439903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.440940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.441231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.441511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.458664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.460420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.464579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:00.475355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:00.477659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:00.479600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 425/866] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:00.490783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.549717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645a338-8105-4d2e-beed-9d3f450fdc03 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.604134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:00.616297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1da1d-782e-4e25-a459-e8ddf59b00c1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.619992 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.621400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.622143 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.622772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.624149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:00.624858 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.628188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.645753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 442/867] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:00.648427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.658632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.659492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.659944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.660326 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.661311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:00.661759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.663348 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.679709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 426/868] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:00.690675 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:00.827746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa4029-2f5c-48d1-b082-a4d5acc3cf07 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.883461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.884727 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.885240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.886056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.887444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:00.887998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.891025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7399b-5425-4b38-98c6-ab365068c623 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.894253 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec3787-80ec-4466-8035-1ef45d789fc0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:00.906560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 443/869] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:00.906560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:00.967556 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:00.968661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:00.969181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:00.969662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:00.971659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=2ca0990c5479452da837df05b9fbcacb) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:00.972227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:00.975360 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966ee27-f231-4888-91c7-68103c9bb1b3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:00.990571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 427/870] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:00 2022] GET /identity/v3/role_assignments?user.id=2ca0990c5479452da837df05b9fbcacb&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:02.063280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69074-5692-4fdb-a309-4fd90c1b807a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:02.064103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69074-5692-4fdb-a309-4fd90c1b807a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:02.064239 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69074-5692-4fdb-a309-4fd90c1b807a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:02.067088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 444/871] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:02.075392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b70b2-a4bf-412e-b149-4d84bef4491b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:02.076092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b70b2-a4bf-412e-b149-4d84bef4491b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:02.076384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b70b2-a4bf-412e-b149-4d84bef4491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:02.105561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a67b70b2-a4bf-412e-b149-4d84bef4491b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:02.107431 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a67b70b2-a4bf-412e-b149-4d84bef4491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:02.137284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:02.245159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 428/872] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:02.284826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69074-5692-4fdb-a309-4fd90c1b807a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:02.455179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69074-5692-4fdb-a309-4fd90c1b807a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:02.492686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:02.493363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:02.493649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:02.493961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:02.494714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:02.496097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0720b50-66ce-4b51-b7e5-4736d740607b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:02.514765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 445/873] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:02 2022] POST /identity/v3/services => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:03.517798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011bab-e76e-4877-b96c-528eadf417d7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.518592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011bab-e76e-4877-b96c-528eadf417d7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.518592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011bab-e76e-4877-b96c-528eadf417d7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.520366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 429/874] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:03.527605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.528518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.528518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.547850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:03.549116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:03.607763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:03.646554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703ab00-ebac-492b-939c-44ff2ea7bf7a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.646831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703ab00-ebac-492b-939c-44ff2ea7bf7a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.647141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703ab00-ebac-492b-939c-44ff2ea7bf7a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.648814 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 430/875] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:03.649784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 446/876] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:03.658437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff803c-7c5a-4741-9fed-edc56c50e972 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.658885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff803c-7c5a-4741-9fed-edc56c50e972 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.659327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff803c-7c5a-4741-9fed-edc56c50e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.676015 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:03.687567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1eff803c-7c5a-4741-9fed-edc56c50e972 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:03.690134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1eff803c-7c5a-4741-9fed-edc56c50e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:03.791351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 431/877] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:03.797334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a43f-0272-47f9-b0a5-edfe64e2ba5d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.798022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a43f-0272-47f9-b0a5-edfe64e2ba5d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.798807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a43f-0272-47f9-b0a5-edfe64e2ba5d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.803489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 432/878] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:03.806318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23f903-bbd8-40b3-b019-8b2074459677 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:03.816255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.816752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.817320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.848752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:03.850429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:03.850930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:03.851259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:03.851543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:03.851800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:03.864817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:03.873227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:03.875065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:03.877244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:03.880167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 447/879] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:03.881123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:03.938836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:03.983790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 433/880] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:03.995195 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.002530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:04.028645 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fc593-ff98-4bb2-a8b3-ab45583a2cb0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.072412 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.073002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.073282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.073561 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.074325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.075703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.110647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 448/881] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:03 2022] POST /identity/v3/services => generated 269 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:04.113147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.178759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9f5b-eb80-403f-b17e-822ab9767d2a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.238402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.239098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.239738 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.240027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.241888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:04.242223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.243667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.254880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 434/882] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:04.260762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.290349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:04.300201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f074f80-2ec6-4616-a449-980edb591f3c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.341502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:04.374186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.375121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.375535 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.375820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.376718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.378807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:04.396234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:04.397422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:04.398352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 449/883] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:04.406191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.411444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67231-e469-4a42-9a44-62a2663aa956 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.462957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.465721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.465721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.465721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.483003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.486271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:04.521334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:04.524112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:04.526308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:04.526308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:04.526308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:04.526308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:04.527755 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 435/884] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:04.537310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.608369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232957-56af-4d2a-aaac-fd0a25973287 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.679790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.681234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.681772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.682454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.684074 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:04.684887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.687853 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.698738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 450/885] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:04.705735 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.737363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:04.748170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6772b-2ecb-4f00-9f8b-44b1533fa006 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.775387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:04.815859 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.816951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.817297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.817826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.819211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:04.819840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.822351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.854086 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 436/886] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:04.855723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:04.880530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33f21-3785-4601-9da9-c6488b1fe623 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:04.946679 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:04.947812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:04.948425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:04.948916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:04.950434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:04.950984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:04.953571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:04.963584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 451/887] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:05.038536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0787e-1b59-48e6-9e18-5206a94ff883 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:05.080811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:05.081742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.082031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.082303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.091635 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:05.093239 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-75163cb9-8ba9-4b88-9b1e-72a68568feb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:05.111696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:05.112743 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:05.113762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 437/888] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:55:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:05.119744 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:05.214959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff83ad-a8cd-4de1-a18d-94c215c72326 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.215140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff83ad-a8cd-4de1-a18d-94c215c72326 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.215470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff83ad-a8cd-4de1-a18d-94c215c72326 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.217114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 438/889] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:05.223648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.224265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.224753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.243306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:05.244829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:05.255503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:05.276870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:05.287197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34f89-089a-4dcb-b54b-0f21b4cc3c18 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:05.333799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 439/890] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:05.354384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:05.356634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:05.358165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.358684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.359186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.360901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:05.361468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:05.362983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:05.384134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 452/891] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:05.390209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:05.539024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e162-f5e1-4240-b813-424d9fc59e8d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:05.581169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:05.582149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.582588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.583042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.588764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb470aa-c44c-4384-b63f-20feb44ec91e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:05.602053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:05.655239 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:05.656541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.657554 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.658083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.658979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:05.660599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:05.673922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:05.675009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:05.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:05.678998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 440/892] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:05.685009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:05.688922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:05.693278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:05.778186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:05.815856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e913-b619-45ba-9cab-db8ef04ce41d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:05.864024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:05.865073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:05.865561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:05.866074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:05.867014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:05.867525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:05.869099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:05.875116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 441/893] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:05.885901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:05.937078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 453/894] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:55:05.943356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.004776 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.005452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.021836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53573d-2b38-4e7d-9bbd-083c15f20400 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.061347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.062481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.063528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.064012 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.064976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:06.065502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.067254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.078203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 442/895] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:06.080488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.086903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb965-d3e9-4843-81cc-743c724f19de None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.150162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.151416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.151877 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.152163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.168248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.170083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:06.194450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:06.195515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:06.196431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:06.196431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:06.196431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:06.196431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:06.200997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 454/896] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:06.202006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.215868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017021df-bb35-499c-9130-fb415b8f2801 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.254705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.255763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.256273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.256768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.257826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] RBAC: Adding query filter params (interface=public, service_id=a3da9a9367ee418e886fa03066dc40e8, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:06.258367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.260041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.271273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 443/897] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/endpoints?service_id=a3da9a9367ee418e886fa03066dc40e8&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:06.273020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.298375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.331036 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.339947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f93b-ff51-47d6-893a-655b7f036280 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.427790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4fae-afd4-4cfc-906e-f6c83ec646e1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.428233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.429716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.430308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.432280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.433888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:06.434409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.438682 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.467428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 455/898] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:06.468567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.489886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.491882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.492414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.492927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.495298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.499404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b49c8-8887-4535-84b8-29dcc64ad1c3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.517409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 444/899] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/services => generated 1268 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:55:06.552924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab39d2-2d48-4cfe-b50e-fd6bbeb0ecf0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.553312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab39d2-2d48-4cfe-b50e-fd6bbeb0ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.553526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab39d2-2d48-4cfe-b50e-fd6bbeb0ecf0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.555238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 445/900] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:06.560967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.561361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.561580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.580083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:06.581377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:06.594271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8efba-c619-4b4a-b7ff-ff5ebeba2381 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.634249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.635068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.635353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.635629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.643941 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.645433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:06.661089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:06.662113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:06.663173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:06.667240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 456/901] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:06.668303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.676302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 446/902] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:06.689406 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.700177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.717530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.789114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf270-ceb3-46f1-b133-1725ba2a72e9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.835131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81535ff9-a7ec-49c6-bca7-b0e1424e3c70 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:06.841414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.842138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.842483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.842773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.843588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:06.845148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.847472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.866814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 457/903] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:06.872636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.883028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:06.883878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:06.884172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:06.884957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:06.885653 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:06.887875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:06.902585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:06.903598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:06.904438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 447/904] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:06.912395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:06.971209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:06.998300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba12e7b-b354-4435-a599-a30f53d2575d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.014313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:07.043790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.044010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6b120-a18f-4160-95b8-cf69066a824d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.044952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.045688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.046205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.048752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] RBAC: Adding query filter params (role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=9c42d851cdef43fcb152473bebeb08f5) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:07.049375 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.051065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.059624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 458/905] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/role_assignments?user.id=9c42d851cdef43fcb152473bebeb08f5&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:07.101707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.102796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.103292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.103774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.104716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:07.105280 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.107009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.113433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 448/906] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:07.119876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:07.242271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235f8d6-a8e4-4c43-a795-6d65f874c9c8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.280813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.282136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.282433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.282715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.283450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:07.283764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.285170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.291520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 459/907] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:07.297581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:07.414287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff83c3-d27c-4fa4-ba49-5153f0763a2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.456002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.456734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.457021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.457344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.458201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] RBAC: Adding query filter params (interface=public, service_id=32f0b17d933b4613a22e6fc777b9e0de, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:07.458521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.460180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac8f23-4a4d-4907-836f-a0883896f154 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.466561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 449/908] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:55:07 2022] GET /identity/v3/endpoints?service_id=32f0b17d933b4613a22e6fc777b9e0de&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:07.472077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:07.476930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:07.482075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd195fb-7979-4a0b-9b58-5f223c2a4b8e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.482075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd195fb-7979-4a0b-9b58-5f223c2a4b8e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.482075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd195fb-7979-4a0b-9b58-5f223c2a4b8e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.482075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 450/909] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:07.487882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.488185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.488455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.501027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:07.507849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:07.509056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:07.585405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 451/910] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:07.608677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:07.610664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0764bc2-8424-4aca-b23a-79e539d4016e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.657791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.658781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.659251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.659832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.660774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.662630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87375ef7-586a-40c4-b052-87f9124d3313 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.670008 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 460/911] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:07 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:07.736640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa865b-b578-4756-aa37-94a3d0795b8e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:07.780398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:07.781105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:07.781428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:07.781739 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:07.782516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:07.783965 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431044a2-c63b-47bb-b58b-81c103f22d68 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:07.788597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:07.807951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 452/912] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:07 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:09.253079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ff98-c4de-4a5c-bbd2-16832e1ffad0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.254167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ff98-c4de-4a5c-bbd2-16832e1ffad0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.254663 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ff98-c4de-4a5c-bbd2-16832e1ffad0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.256582 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 461/913] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:09.262372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fda061-c3be-436c-a288-ff918552aeec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.263298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fda061-c3be-436c-a288-ff918552aeec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.263816 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fda061-c3be-436c-a288-ff918552aeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.283022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-95fda061-c3be-436c-a288-ff918552aeec None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:09.284568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95fda061-c3be-436c-a288-ff918552aeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:09.411125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 453/914] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:09.434038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ff98-c4de-4a5c-bbd2-16832e1ffad0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:09.511128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:09.549597 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ff98-c4de-4a5c-bbd2-16832e1ffad0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:09.587468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:09.588260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061c372-a756-44e6-a28f-74dae441ed2f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.588387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.588561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061c372-a756-44e6-a28f-74dae441ed2f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.588655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.588831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.588926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061c372-a756-44e6-a28f-74dae441ed2f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.590531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 454/915] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:09.598401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.599001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.599483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.601341 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:09.610610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:09.611643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:09.612499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 462/916] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:09.618127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:09.622553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:09.624107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:09.734307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 455/917] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:09.740176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10737f-41d1-440b-b018-47de9e4261e5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:09.758649 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:09.780269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:09.780968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.781239 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.782346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.783262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:09.783753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:09.785435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:09.795968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 463/918] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:09.797927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:09.802985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:09.862594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:09.920342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b21648-e2d6-46f7-b004-8c4c211ef68b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:09.961930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496e99-801b-4a05-8ba7-dcdc67116004 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:09.976594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:09.977900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:09.978364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:09.979137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:09.979666 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:09.981876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-14aaebce-4b83-4dfb-83fd-0096e4ed898b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:10.002913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:10.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:10.006982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 456/919] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:10.008611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df5ec-49ad-457c-8e5c-eb032d1a64db None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.008611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df5ec-49ad-457c-8e5c-eb032d1a64db None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.008611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df5ec-49ad-457c-8e5c-eb032d1a64db None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.010431 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 457/920] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:10.016876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df5ec-49ad-457c-8e5c-eb032d1a64db None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.040085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.041828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.042738 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.043247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.064237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.066795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6e8c7ade-65e0-4876-bc99-26a300fce43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:10.088342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:10.089445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:10.090600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:10.090600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:10.090600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:10.090600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:10.094911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 464/921] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:10.096200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.096489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.096763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.115058 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:10.116287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:10.145480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df5ec-49ad-457c-8e5c-eb032d1a64db None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.165451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.182532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.189521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.190283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.190571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.190936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.191973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:10.192324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.194023 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.205717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 458/922] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:10.205717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.242437 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 465/923] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:10.243134 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.362394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a1846-86b3-4b11-92bf-14bd01e212fb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.373520 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c6e9a-0083-4558-91ca-d078fcd3dc4a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.411674 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.412377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.412650 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.412930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.413652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:10.414002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.415396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.425927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 466/924] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:10.425927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.428896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.429956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.430481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.430948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.431835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:10.432193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.434549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.455342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 459/925] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:10.456190 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.488071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.513512 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.598232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1378146-e5c1-4e30-a87c-e4b97d7d26f1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.602882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefcd45-e711-45a0-a351-dd54f39309fe None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.657965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.658170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.658897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.659165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.659357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.659625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.659912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.660101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.661168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.661372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.663560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.663726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa483ff-3c92-4a62-bb19-95793cf08fb2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:10.680420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:10.681648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:10.683199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 467/926] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:10.684559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 460/927] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc231-e906-4331-912c-cf1a935b577f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc231-e906-4331-912c-cf1a935b577f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc231-e906-4331-912c-cf1a935b577f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 461/928] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:10.705903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc231-e906-4331-912c-cf1a935b577f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.858989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.866180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f71f0-8761-46db-a29e-17c6fcedf2af None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.874421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:10.893917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc231-e906-4331-912c-cf1a935b577f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:10.929296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.931591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.931591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.931591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.940241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:10.940678 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.941288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.941776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.942122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.942352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.943323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:10.943796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:10.945805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c041cf-c2fa-4c73-a480-e4d8a007e215 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:10.952188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 462/929] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:10.953437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:10.954033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:10.954226 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:10.963733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:10.965172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:10.966946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:10.968248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 468/930] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:10.970437 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:10.976244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:10.977562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:11.058662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 463/931] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:11.059882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.160702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758283d-6b9c-4c13-97c0-6609b5c73a66 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.178361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.198480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67790-8a91-4bfc-bd6e-5626e89dcd8b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.217373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.218614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.219676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.220189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.221905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:11.222163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.224799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.241309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 469/932] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:11.241309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.261353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.262247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.262525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.262805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.264098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:11.264421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.266146 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.286918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.287119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 464/933] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:11.287810 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.401135 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff6c8-0857-4820-9118-d021ffb76200 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.428031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6ce26-421c-4524-8d01-63214945b04c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.447902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.448805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.449026 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.449426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.449981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.451373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:11.465834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:11.467190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:11.468084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 470/934] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:11.469426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.479804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.480630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.480938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.481393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.482401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.484161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.504203 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.511769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 465/935] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:11.514183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.608329 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.628331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904bf47-fdd2-4f4a-8cbf-1e855e1a885d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.698709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.699961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.701159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.701741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.704850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1fbd5-634f-4657-a3d7-0b085fbcf52e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.720786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.722848 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:11.744224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:11.752791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 471/936] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:11.756185 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.757001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.757001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.757001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.757001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.757590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:11.757590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.757590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.764355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 466/937] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:11.771732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.858262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.897847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031a1-02d0-467f-8c26-0df5ce544b17 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.903085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dbbd8-b2d8-4917-a40a-9df514a7f7fb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:11.945120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.946315 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.946811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.947277 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.948615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:11.949194 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.951060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.953188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:11.954858 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:11.955178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:11.955607 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:11.956777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:11.957289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:11.959203 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:11.959677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 467/938] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:11.966198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:11.967064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:11.979386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 472/939] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:11.984935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:12.093794 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0de32-09a0-4db6-aa99-5aa2ab5ed58a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:12.110513 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97473654-47a7-4877-b0ab-2a54a167fc99 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:12.135760 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:12.152553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:12.153378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.153661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.154122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.156011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:12.157306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.158053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.158574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.160078 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] RBAC: Adding query filter params (interface=public, service_id=eec0b7f2daa7445697087a130d414ef2, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:12.160659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:12.162532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:12.163510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:12.165039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:12.176662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 468/940] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/endpoints?service_id=eec0b7f2daa7445697087a130d414ef2&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:12.178305 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:12.180623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:12.182343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:12.184175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:12.187079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 473/941] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:12.188810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:12.250497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:12.301344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1effd-89bb-45a9-972a-d968a4a6a9cf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:12.338870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:12.339544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.339832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.340113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.340925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:12.341237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:12.342764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:12.356720 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcd7a0-c9ba-4b6e-a1bc-9bd3ee747d1e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:12.362034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 474/942] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:12.364510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:12.407448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:12.409040 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.409818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.410501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.411977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:12.413894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b9821-f428-4507-9b49-3db1845ff5a6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:12.429820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 469/943] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:12 2022] GET /identity/v3/services => generated 1497 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:12.488836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4690-729d-4a5a-9f7f-4ad18a617650 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:12.511515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:12.532775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:12.533822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.534110 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.534392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/9c42d851cdef43fcb152473bebeb08f5/roles/a2d2bc3f0a524c48bc9e42c326ea6fc0` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.536951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=9c42d851cdef43fcb152473bebeb08f5, role_id=a2d2bc3f0a524c48bc9e42c326ea6fc0)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:12.538499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe8492-cd21-4458-8b97-0220e32bec7d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:12.558568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 475/944] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:55:12 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/9c42d851cdef43fcb152473bebeb08f5/roles/a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:55:12.911735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969a985-0410-40d4-93ca-5b0a5118a961 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.911988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969a985-0410-40d4-93ca-5b0a5118a961 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.912329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969a985-0410-40d4-93ca-5b0a5118a961 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.913856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 470/945] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:12.919499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:12.919791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:12.920069 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:12.939720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:12.941346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:13.014454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 476/946] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:13.038980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969a985-0410-40d4-93ca-5b0a5118a961 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:13.133231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:13.222504 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969a985-0410-40d4-93ca-5b0a5118a961 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:13.284771 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:13.285800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:13.286240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:13.286655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:13.287870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:13.290633 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:13.293174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING keystone.server.flask.common [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 11 11:55:13.326995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.sql.core [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:55:13.326995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 11:55:13.326995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [parameters: {'id': '12811b34e5ab4eeb86f16e5a9b1791f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 11:55:13.326995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80222) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12811b34e5ab4eeb86f16e5a9b1791f8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12811b34e5ab4eeb86f16e5a9b1791f8. Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:55:13.362298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result.read() Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 11 11:55:13.363201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application yield resource Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise self.value Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application session.commit() Jul 11 11:55:13.364150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:55:13.365118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:55:13.366093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result.read() Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:55:13.366921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [parameters: {'id': '12811b34e5ab4eeb86f16e5a9b1791f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:13.367790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:13.368660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12811b34e5ab4eeb86f16e5a9b1791f8. Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:13.369640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 471/947] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 11 11:55:13 2022] POST /identity/v3/projects => generated 262 bytes in 327 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:13.380586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:13.486877 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df1c3c-b030-4098-b854-87aaeee918f6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:13.495209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:13.525621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:13.526798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:13.527310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:13.527803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:13.535120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:13.536904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-21ca916b-4c95-4837-8b16-edab3a319c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:13.551315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:13.552287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:13.553236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:13.555775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 477/948] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:13.562007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:13.731109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a7f0-8c6b-4113-9a28-0bdad73d6bcc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:13.744587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638de8d8-dcc0-43f1-a4a5-10ac784e4455 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:13.744922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638de8d8-dcc0-43f1-a4a5-10ac784e4455 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:13.745168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638de8d8-dcc0-43f1-a4a5-10ac784e4455 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:13.751059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 478/949] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:13.751819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:13.752947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f89f-707d-4428-81d6-80e435722430 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:13.753199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f89f-707d-4428-81d6-80e435722430 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:13.753540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f89f-707d-4428-81d6-80e435722430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:13.772670 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7062f89f-707d-4428-81d6-80e435722430 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:13.774016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7062f89f-707d-4428-81d6-80e435722430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:13.785283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:13.785993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:13.786280 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:13.786557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:13.787325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:13.788107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:13.790148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:13.818644 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 472/950] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:13.869859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 479/951] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:13.888976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:14.007784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1224d0-2144-457a-a10e-f3b15a7cdafd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:14.044514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:14.045337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.045625 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.045935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.046604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:14.048565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:14.061524 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:14.063498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:14.064664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:14.068668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 473/952] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:14.069730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062f89f-707d-4428-81d6-80e435722430 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:14.186706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:14.195277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062f89f-707d-4428-81d6-80e435722430 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:14.240475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:14.241156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.241440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.241745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.242470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:14.242779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:14.244327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7320-f2f3-4dea-bf7a-c2a3c4705a8d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:14.250166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 480/953] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:14.259294 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:14.317618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:14.438552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0289b6-014c-449a-b70c-b4367ff68ac1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:14.491011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:14.492141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.492652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.493151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.494359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:14.494901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:14.497175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ac512-d4e4-4983-9dfe-c413c11e6f6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:14.506201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 474/954] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:14.833356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6ab6-4e7b-4783-aa73-64435732aa75 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.834668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6ab6-4e7b-4783-aa73-64435732aa75 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.835207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f6ab6-4e7b-4783-aa73-64435732aa75 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.837128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 481/955] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:14.843285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.844829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.845325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.863913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:14.865925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:14.941483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 475/956] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:14.965245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f6ab6-4e7b-4783-aa73-64435732aa75 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.096842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f6ab6-4e7b-4783-aa73-64435732aa75 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.139449 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.140275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.141141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.141434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.142160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:15.143703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a20e9706-a2d6-4b20-9eb5-89207ed23c96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:15.157281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:15.159400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:15.161920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 482/957] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:15.169835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.242194 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:15.282215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8023c2-6b0c-4ce2-9fa6-8bcb2958da4a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.282715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8023c2-6b0c-4ce2-9fa6-8bcb2958da4a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.283100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8023c2-6b0c-4ce2-9fa6-8bcb2958da4a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.286584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72e30-24e4-40c6-910c-b4155023e1ae None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.291653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 483/958] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:15.295673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.295673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.295673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.319684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:15.321246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:15.327016 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.327942 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.328962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.329409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.330340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:15.330805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:15.332373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:15.338604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 476/959] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:15.345276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.348378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:15.410219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 484/960] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:15.439547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.483588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:15.511675 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2cbd1-7dbd-4516-aef6-2b725155d7f8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.581779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.582601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5d74d-44d8-41f9-96b7-e601937c3b2a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.583193 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.583696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.584833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.586983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:15.587582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:15.589268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:15.602894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 477/961] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:15.604877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.630912 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.632072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.632072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.632357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.645762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:15.654749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:15.655798 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:15.656769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:15.661386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 485/962] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:15.662609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.684296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:15.777799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551ac98-8cc1-48b6-99aa-a5d64bba5e4b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.823998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e67b-1825-4f8a-bbb0-6cf52f9d9dab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:15.830997 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.832042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.832594 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.833225 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.834235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:15.836350 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:15.873839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 478/963] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:15.912975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:15.915371 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:15.916125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:15.916880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:15.918636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:15.919510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:15.923198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:15.941624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 486/964] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:15.952042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:15.981340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.143312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808ad3-8d73-4c8c-875c-6677df17f832 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.196844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.197664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.197984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.198265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.211370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.212866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a6a5177c-a315-4954-a1bb-2f2ca685ca49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:16.232520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:16.233518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:16.235062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:16.235062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:16.235062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:16.235062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.235062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 479/965] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:16.241856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.270658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac833f0-12e9-43d1-a7df-9a3bb86823a5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.271207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac833f0-12e9-43d1-a7df-9a3bb86823a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.271501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac833f0-12e9-43d1-a7df-9a3bb86823a5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.273249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 480/966] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:16.278829 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.279124 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.279402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.298749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:16.300019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:16.340626 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.357598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550228-efbe-4c5d-99d5-1c25e38d5d66 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.362725 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.381733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 481/967] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:16.398928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.399540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.399831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.400115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.400860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:16.400973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.401174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.402592 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.415977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 487/968] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:16.421478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.534787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd1ca3-d8a1-4f33-bc1d-54613a8b6408 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.544319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc85e61-ff81-45d4-b084-8c3d5dc06ce5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.577251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.578010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.578304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.578648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.578757 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.579602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.581210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.586094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.587205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.587717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.588218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.598001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.sql.core [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:55:16.598001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 11:55:16.598001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [parameters: {'user_id': 'a2376bbfc9dd4a3db6f028dc69836120', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 11:55:16.598001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80222) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:55:16.598791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.600640 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result.read() Jul 11 11:55:16.607632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:55:16.608631 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application yield resource Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise self.value Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application session.commit() Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:55:16.610474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:55:16.612166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:55:16.613440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application result.read() Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:55:16.616251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a2376bbfc9dd4a3db6f028dc69836120', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:16.619067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 11 11:55:16.622811 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 482/969] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 11 11:55:16 2022] POST /identity/v3/users => generated 160 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:16.629261 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:16.637649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:16.639560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:55:16.641480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:16.646198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 488/970] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:16.655627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.684126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.735952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028054b-a985-4ec9-b1f7-d7ba50ec5e23 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.774233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.775187 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.775489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.775751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.786545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.788868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:16.806278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:16.808198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:55:16.811263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:16.813327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 483/971] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:16.813746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:16.860593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549432b-1f72-4d38-b25b-6a4c49cd3ac5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.926347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.927303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f11fa-e720-4bab-8b72-801538fd6261 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:16.929514 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.930421 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.931249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.933336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:16.934340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.937451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.943420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:16.968163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:16.969763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:16.970235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:16.970682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:16.971663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:16.972127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:16.973753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59e6a-a355-45bb-90c9-2003faaf2e9b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:16.982211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 489/972] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:16.986090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:17.021833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 484/973] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:17.087349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:17.128058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a21d5-eab9-4abc-b8d9-be5009ecc1e0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:17.128058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a21d5-eab9-4abc-b8d9-be5009ecc1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:17.128058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a21d5-eab9-4abc-b8d9-be5009ecc1e0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:17.128058 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 485/974] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:17.137385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:17.137385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:17.137385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:17.160151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:17.162287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:17.188276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a3d5-a0c7-4d1f-9406-1b41c1e7c300 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:17.253911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 486/975] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:17.259630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:17.260364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:17.260639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:17.260910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:17.273914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:17.380533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:17.382358 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:17.440318 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea0d92-cc27-4fd1-9c47-5e2fe94fff73 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:17.442499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:17.450633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:17.486185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:17.486867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:17.487149 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:17.487427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:17.488174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:17.489570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:17.517719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 487/976] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:17 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:17.541692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 490/977] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:16 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:55:17.551184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:17.731462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc6299-2bae-4888-809d-658a3d73e13b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:17.773248 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:17.774293 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:17.774707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:17.775114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:17.787974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:17.790180 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:17.811071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:17.813578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:17.815430 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:17.815430 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:17.815430 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:17.815430 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:17.815430 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 488/978] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:17.822307 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:17.958879 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8363be-cf3f-44bf-9801-c466d402ce9f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.009480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.010539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.010909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.011197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.012200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:18.012571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.014674 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.037772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 491/979] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:18.041489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:18.059525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:18.197569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbacb-1100-42d3-961d-e6f6d3af309e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.245377 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.246379 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.246818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.247233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.256731 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.258693 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-72951a65-88b3-40a4-b124-d15893de06d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:18.278468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:18.286142 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:18.288249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 489/980] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:18.294765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:18.321489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:18.348536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d6aa8-10bb-4605-83ca-a2c9fc010eb8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.348861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d6aa8-10bb-4605-83ca-a2c9fc010eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.349107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d6aa8-10bb-4605-83ca-a2c9fc010eb8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.350617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 490/981] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:18.356279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.356570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.356891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.376611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:18.377882 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:18.457566 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:18.465567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 491/982] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:18.467620 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aee80-2717-4659-8bb1-11fa5267cb3d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.489236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:18.518440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.519565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.520133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.520420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.521490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:18.522249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.523976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.539406 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 492/983] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:18.545400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:18.624930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:18.644528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a1ad8-809d-4c66-a8e9-9483ccc255d2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.661135 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7281b-3699-4958-8b84-b89c0b12baeb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.706199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.708038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.708784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.709769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.709927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.710892 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.711364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.711832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.712196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.713308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] RBAC: Adding query filter params (role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=9c42d851cdef43fcb152473bebeb08f5) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:18.713843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.715421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.716258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.726189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 493/984] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/role_assignments?user.id=9c42d851cdef43fcb152473bebeb08f5&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:18.738143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:18.739985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:18.742371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 492/985] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:18.747970 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:18.914956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9da99-a1ff-429c-87d5-23d005252e31 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:18.967294 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:18.968336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:18.968784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:18.969255 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:18.970493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:18.970819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:18.972447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:18.974522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:18.982896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 494/986] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:18.988329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:19.057042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:19.101818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50036a24-0ad7-498f-919b-86655e6f5194 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:19.151745 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:19.152741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:19.153178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:19.153632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:19.154520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:19.154975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:19.156526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:19.162863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 493/987] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:19.173263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:19.350498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b2ce8-4fe7-4c94-87ef-83e517f6e558 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:19.408295 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:19.409417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:19.409956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:19.410452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:19.411925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] RBAC: Adding query filter params (interface=admin, service_id=eec0b7f2daa7445697087a130d414ef2, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:19.412467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:19.414997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36520-e6bd-4664-96b6-52c61f837108 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:19.429751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 495/988] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Jul 11 11:55:19 2022] GET /identity/v3/endpoints?service_id=eec0b7f2daa7445697087a130d414ef2&interface=admin®ion_id=RegionOne => generated 193 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:19.431199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:19.548013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a7d0-4250-413f-acba-6a3d568c7448 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:19.577635 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:19.588268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:19.589085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:19.589522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:19.589988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:19.590808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:19.592476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347ffa1-e5f7-4625-8e80-cf33b33b1ef3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:19.606135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 494/989] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:19 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:19.825728 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab0df-1f48-4089-bac4-51ef78bda0bc None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:19.826033 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab0df-1f48-4089-bac4-51ef78bda0bc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:19.826322 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab0df-1f48-4089-bac4-51ef78bda0bc None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:19.828423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 496/990] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:19.833702 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:19.833994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:19.834285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:19.854754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:19.856736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:19.981727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 495/991] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:20.004472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ab0df-1f48-4089-bac4-51ef78bda0bc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:20.087401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:20.149495 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ab0df-1f48-4089-bac4-51ef78bda0bc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:20.192055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:20.192940 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.193253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.193548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.194250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:20.195643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:20.208889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:20.210895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:20.211968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 497/992] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:20.216372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:20.336151 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedd86c-8c22-436b-8adb-5a25ea132ae2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:20.361491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:20.377081 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:20.378215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.378729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.379400 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.380470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] RBAC: Adding query filter params (name=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:20.380972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:20.382941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:20.394045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 496/993] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:20.395970 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:20.559501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d120c97-c51c-40f4-9f0f-b22656a05b4b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:20.592054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:20.603253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:20.604098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.604521 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.604926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.605839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:20.606272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:20.607791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:20.619360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 498/994] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:20.620615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:20.750482 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cec06-3c56-4157-93db-45a2657efcda None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:20.797956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:20.798621 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.798920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.799192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.799998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] RBAC: Adding query filter params (interface=public, service_id=c95fd7c0da874e5391382111182f54e7, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:20.800316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:20.801733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29c4fd-bb81-470b-9251-8dd2a89428ca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:20.811886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 497/995] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:55:20 2022] GET /identity/v3/endpoints?service_id=c95fd7c0da874e5391382111182f54e7&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:20.814120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:20.887991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa0e9-ff32-463d-af96-bd26a0d1c667 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.888167 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa0e9-ff32-463d-af96-bd26a0d1c667 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.888445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa0e9-ff32-463d-af96-bd26a0d1c667 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.890206 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 498/996] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:20.896851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:20.897228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:20.897712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:20.915751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:20.916954 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:20.972026 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9874e7-8c34-432a-aefe-8cd5ef598cb6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:20.979678 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:21.006876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 499/997] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:21.029170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:21.032075 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:21.032775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.033052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.033363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.034068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:21.035460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:21.042576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 499/998] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:20 2022] GET /identity/v3/services => generated 1727 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:21.177300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c986-ebc7-4895-aeb6-b459c2b96cbd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:21.222975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:21.223798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.224081 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.224362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.225036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:21.226483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:21.242390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:21.244325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:21.245271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 500/999] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 11 11:55:21 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:21.255191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:21.326042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:21.388458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d02-140c-4fe4-aa3f-70fadb7260c1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:21.427628 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:21.428217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.428500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.428782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.429493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:21.429835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:21.431204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c223-ba33-4c59-8ac6-e6488fd233d8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:21.436658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 500/1000] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:21.443628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:21.524218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:21.601110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345380-dd89-4326-b0b0-500455975194 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:21.645546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:21.646270 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.646855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.647562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:21.647869 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:21.649255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159fa4b-fe60-43d1-90ab-d56b91530aeb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:21.661711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 501/1001] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:21.721243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35054c-e837-429d-9aa7-74587944ece1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.721555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35054c-e837-429d-9aa7-74587944ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.721809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35054c-e837-429d-9aa7-74587944ece1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.723449 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 501/1002] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:21.729504 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:21.729861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:21.730135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:21.752235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:21.753665 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:21.854085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 502/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:21.873284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35054c-e837-429d-9aa7-74587944ece1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:22.011888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35054c-e837-429d-9aa7-74587944ece1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:22.036543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:22.058299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:22.059128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:22.059415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:22.059694 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:22.060356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:22.061849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:22.075277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:22.076393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:55:22.077250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:22.081829 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 502/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 11 11:55:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:22.083157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:22.143676 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:22.225158 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0e20e-65ae-4c11-9496-7c5300c7deb1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:22.284993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:22.286106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:22.286617 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:22.287101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:22.288428 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:22.288988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:22.291648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:22.299207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 503/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:22.305499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:22.437352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1d75e-b958-410f-98a7-0551858f07a3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:22.489277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:22.490190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:22.490481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:22.490761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:22.491504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:22.491958 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:22.494127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fd02e-a9c8-488e-9790-51541cc509de None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:22.502961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 503/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 11 11:55:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:22.510134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:22.697530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81010791-dafd-40e0-89f1-9649f08cec8e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:22.699819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:22.760025 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:22.761185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:22.761723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:22.762158 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:22.763533 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:22.765585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598636b-90f5-441d-a029-e2814a991a16 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:22.790161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 504/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:23.191828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43d04-bd31-4ec3-b29c-f6c290ece608 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.192525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43d04-bd31-4ec3-b29c-f6c290ece608 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.192525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43d04-bd31-4ec3-b29c-f6c290ece608 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.194165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 504/1008] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:23.199875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.200256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.200429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.221957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:23.223186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:23.337726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 505/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:23.371795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb43d04-bd31-4ec3-b29c-f6c290ece608 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:23.433819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:23.498498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb43d04-bd31-4ec3-b29c-f6c290ece608 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:23.538504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:23.539393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.539685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.539924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.540548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:23.541996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:23.555379 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:23.557297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:55:23.559894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:23.568990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 505/1010] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:23.569215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:23.720645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:23.737590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153326-9eb2-4b1f-af3f-0fa911907320 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:23.777590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:23.778297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.778584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.778865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.779585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:23.779920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:23.781346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148215c6-ff5f-455c-9ef4-3b72436eecc4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:23.787339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 506/1011] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:23.793257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:23.909001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc35cf-ec40-482e-aab3-9d11ac4d231c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:23.920630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:23.925641 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0ccd-af4c-4757-a2af-8045b7228672 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.926022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0ccd-af4c-4757-a2af-8045b7228672 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.926273 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0ccd-af4c-4757-a2af-8045b7228672 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.927847 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 507/1012] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:23.932858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.933179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.933453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.950714 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:23.951396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.951681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.951967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.952204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:23.952665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:23.953014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:23.953798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:23.954424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:23.963372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 506/1013] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 11 11:55:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:23.969720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:24.065298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 508/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:24.092842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:24.132813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bbd76-202f-45cf-921f-f37e3b07de23 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:24.184123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:24.185188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:24.185662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:24.186164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:24.187441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:24.189704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86286-2542-49b4-a5ba-4c3e29957a69 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:24.222728 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 507/1015] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:24.224048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:24.248304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9127b9-c688-417b-8ba7-bd3cde9fe404 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:24.300580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:24.301591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:24.302060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:24.302467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:24.303692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:24.306342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa6373-0166-4f3a-85da-02ee7c18771a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:24.323106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 509/1016] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:24 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:25.005992 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9dd53-c7ef-4b8d-b703-2bd4194b62a4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.006271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9dd53-c7ef-4b8d-b703-2bd4194b62a4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.006672 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9dd53-c7ef-4b8d-b703-2bd4194b62a4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.007953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 508/1017] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:25.014223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.015020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.015020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.035160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:25.036369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:25.147649 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 510/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:25.176910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9dd53-c7ef-4b8d-b703-2bd4194b62a4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:25.234507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:25.362778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9dd53-c7ef-4b8d-b703-2bd4194b62a4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:25.413716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:25.414928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.415433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.415954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.418178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:25.420697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c840e-2835-459b-8de0-ff6f4e4929a5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:25.424477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 509/1019] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:25.430692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:25.508382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:25.572672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634430-3cbd-45a9-82b9-3b573f1b901b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:25.615988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:25.616572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.616864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.617147 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.617911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:25.619394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672c4a4-392c-4dd8-a9a9-e33924595796 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:25.652219 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 511/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:25 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:26.425035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d345f-7750-4863-b5ae-d25b2185fb20 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.425557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d345f-7750-4863-b5ae-d25b2185fb20 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.425557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d345f-7750-4863-b5ae-d25b2185fb20 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.429702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 510/1021] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:26.432701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc96e7-c7b2-462e-8a7b-e32574abc0c5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.433338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc96e7-c7b2-462e-8a7b-e32574abc0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.433338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc96e7-c7b2-462e-8a7b-e32574abc0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.451766 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1acc96e7-c7b2-462e-8a7b-e32574abc0c5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:26.453045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1acc96e7-c7b2-462e-8a7b-e32574abc0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:26.529174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 512/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:26.553397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d345f-7750-4863-b5ae-d25b2185fb20 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.633182 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591abf1-7940-4c73-b5b6-a1f2568afbee None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.634527 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591abf1-7940-4c73-b5b6-a1f2568afbee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.635033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591abf1-7940-4c73-b5b6-a1f2568afbee None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.637049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 513/1023] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:26.644361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.644647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.644931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.665875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:26.668071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:26.674400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:26.680824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d345f-7750-4863-b5ae-d25b2185fb20 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:26.728342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:26.729182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.729466 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.729768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.748445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:26.757708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:26.759554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:26.761817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 511/1024] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:26.764843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:26.766105 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.809445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 514/1025] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:26.850444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.883861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa5d9-d138-486a-8692-2ad3e5082e01 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:26.929754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:26.930444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.930734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.931072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.931785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:26.932099 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:26.933915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:26.945621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 512/1026] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:26.951121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.977440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:26.979085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3492cdd-0e16-4c18-a9e9-87f9478fce8c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.050831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.052092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.052549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.052997 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.054404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.056705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:27.072951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:27.074906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:27.075861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 515/1027] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 11 11:55:26 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:27.081964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.095747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a879c51-f51b-49fb-b8da-930a49486917 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.141014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.141879 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.142169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.142481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.149233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.158137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.160526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:27.182393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:27.183415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:27.185304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:27.185304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:27.185304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:27.185304 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:27.189723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 513/1028] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:27.190439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.244074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fbc41-1a07-4263-b7da-0a8ef0da2045 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.273343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.295429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.296929 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.297447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.297985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.298832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.299399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.301653 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.307716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 516/1029] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:27.311445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e198d-6787-49c6-a732-6c990a35f5a9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.313701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.362194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.362882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.363171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.363447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.364171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.364506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.365957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.379646 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 514/1030] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:27.386071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.440453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30edc-93ac-450f-a718-bc7fc1a99a99 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.442898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.486344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.487908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.488673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.489395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.490975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.491759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.494636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.504740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 517/1031] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:27.510996 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.543468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5a6a-9eb2-43a8-b2fa-1f02caab3926 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.580897 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.587515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.588427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.589109 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.589434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.599557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.602454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:27.621444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:27.622742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:27.629664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 515/1032] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:27.630573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.662082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0afeb-332f-4ebb-bdae-fd58eb963d2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.724177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.725145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.725622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.726129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.727616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] RBAC: Adding query filter params (interface=public, service_id=c083b0f3a1ca4721be14310597f5ead8, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.728172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.730837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.745297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 518/1033] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/endpoints?service_id=c083b0f3a1ca4721be14310597f5ead8&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:27.747539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.808101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.812688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680618f-f5f7-4387-a36f-f3f94dd2941a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.879995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.881073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.881575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.882136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.883242 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.883710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.885360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.911089 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 516/1034] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:27.912285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.918355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bcac-47bb-45e9-89df-0fece3742005 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.957772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.958614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.959027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.960212 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.961643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.963062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669745bc-615c-450f-b251-eefc24d7b39e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.971614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 519/1035] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/services => generated 1957 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:27.972817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f24f-48f7-41e1-a8d2-af8187e50090 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.973116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f24f-48f7-41e1-a8d2-af8187e50090 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.973399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f24f-48f7-41e1-a8d2-af8187e50090 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.974856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 520/1036] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:27.975657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.989708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.990312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.990818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.010027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:28.011530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.062477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2542a2-2251-4bac-8941-f56c012f8247 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.110733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.111526 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.111887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.113819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.114013 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 521/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:28.137947 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.174639 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:28.225666 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.228391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.272304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b264-9d79-40fd-993c-6ff71c1a12e4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.297087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:28.335497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.337285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.337845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.338283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.340446 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.344346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.352673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 522/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:28.355047 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.369003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 517/1039] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:28.370677 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.488971 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:28.507157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5654ae-b1ca-4a1c-a78e-d733408ba925 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.536041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93772506-8518-47ef-9e17-dce35c3d112b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.549653 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.550369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.550656 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.550944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.551684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.553137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.581271 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 523/1040] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:55:28 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:28.602959 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.604345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.604724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.605229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.626569 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.629492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:28.661176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:28.663246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:28.665021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:28.665021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:28.665021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:28.665021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:28.665021 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 518/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:28.669795 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.828022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e50b1-5c78-4361-b34a-bdbf90f5ba33 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.872465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.873177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.873470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.873798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.874589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:28.874904 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.876335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.893083 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 524/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:28.898913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.902142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:29.025003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1933cc6-2b3a-4c06-aac5-da3ee020a6ad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.079929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.080643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.080957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.081238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.091988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.095215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:29.114756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:29.126562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:29.128388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 519/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:29.129365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.148690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:29.247517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33b39d-6b9c-4726-97a5-1520f00f0304 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.283622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.284300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.284577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.284857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.285652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:29.285988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.287409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef432d32-c608-48c9-9271-68fe3e0b6dc9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.305006 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 525/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:29.306962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.373063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:29.425666 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18144bc-325b-4633-a00a-4d20f9f3dab8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.461470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.462166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.462459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.462741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.463571 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] RBAC: Adding query filter params (role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:29.463890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.465325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc197cd-2957-43b7-b67d-c6e615c8783f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.477982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 520/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:29 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:30.183392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43162c4-3d71-47bf-8c55-8290f17c963d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.183911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43162c4-3d71-47bf-8c55-8290f17c963d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.183990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43162c4-3d71-47bf-8c55-8290f17c963d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.185357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 526/1046] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:30.190932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.191400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.191475 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.208366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:30.209984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:30.280064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 521/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:30.305401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43162c4-3d71-47bf-8c55-8290f17c963d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.383548 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.417842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43162c4-3d71-47bf-8c55-8290f17c963d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.451516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.452311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.453228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.453563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.454250 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.455613 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:30.467673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:30.469398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:55:30.470655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:30.474030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 527/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 11 11:55:30 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:30.475575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.533511 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.581906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514f09a-bc0e-459c-95df-225844f14be1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.618716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.619393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.619685 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.619963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.620680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] RBAC: Adding query filter params (name=image) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:30.620993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.622400 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.632293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 522/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:30.633335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.741187 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e9e3-5ce4-43f6-9eec-96ca10c834aa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.785429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.786136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.786520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.786805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.788074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:30.788074 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.789974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c58c7-46d4-42e4-81d8-7e90a8b76d75 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.791502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.803032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 528/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:30.804194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.914070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b641a7-88a7-4cdf-a2f9-5012ee99979b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.936515 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.960055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.960725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.961054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.961335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.962150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.963537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72762-befc-41dd-a69c-8de3e1d95755 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.987549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 523/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:55:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:31.034654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b029b-ea78-44df-9c33-af90617f6e7c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.034866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b029b-ea78-44df-9c33-af90617f6e7c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.035161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b029b-ea78-44df-9c33-af90617f6e7c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.036863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 529/1052] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:31.042913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.043246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.043501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.061653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:31.063480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:31.161723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 524/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:31.185778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b029b-ea78-44df-9c33-af90617f6e7c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.308220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b029b-ea78-44df-9c33-af90617f6e7c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:31.348363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:31.349160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.349450 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.349749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.371059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8c59c2df-d89b-48dc-a8b9-c2877872b26e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:31.379691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:31.380812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:31.381742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 530/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:31.387204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.501491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d387d-868c-4490-baa6-647501cb71e5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:31.503127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:31.547361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:31.548519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.549019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.549525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.550572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:31.550890 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:31.552435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:31.567551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 525/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:31.567742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a93d-68c2-4db4-986e-03d3af048885 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.567742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a93d-68c2-4db4-986e-03d3af048885 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.567742 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a93d-68c2-4db4-986e-03d3af048885 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.568168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 531/1056] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:31.568720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.575224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.575516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.575773 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.594562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:31.597027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:31.657515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:31.676578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 532/1057] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:31.695905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cdd1e-17c6-4b6f-97c8-60767c6140ad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:31.700488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.736351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:31.737270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.737560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.737868 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] PATH_INFO: `/v3/users/1877b3fdea9d4a44b2179895a87ecf80` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.756759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] RBAC: Authorizing `identity:get_user(user_id=1877b3fdea9d4a44b2179895a87ecf80)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:31.758252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:31.760222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 526/1058] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/users/1877b3fdea9d4a44b2179895a87ecf80 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:31.766001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.795604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:31.818461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23109e12-17ad-4c9f-9c88-f0ae14e4a585 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:31.859861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:31.860680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.860977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.861255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.872174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:31.881632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:31.882687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:31.883552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 533/1059] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:31.889327 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:31.893968 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d1cc7-07db-484a-910f-ab0505baf29d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:31.937097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:31.938031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:31.938505 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:31.939019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:31.942586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.008270 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f9b9-ed6c-43e5-b8bc-93fbe5d987cd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.022962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.024792 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.051064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.056419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.057580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.058141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.058619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.060022 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:32.060596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.063368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.079097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 534/1060] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:32.086491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.155508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 527/1061] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:32.162308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.239650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b182a1-7171-42ca-9167-815100c2a039 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.252803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.283926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.284748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.285028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.285308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.297372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.298862 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.302754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289f84e-e71d-4e68-8b24-f539ac069957 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:32.318662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:32.319704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 535/1062] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:32.330349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.345974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.361237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.362140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.362474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.362770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.363997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.365435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.372073 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 528/1063] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:32.373363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.448958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6abb5-ad1a-44e1-a620-a77a55a404f4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.491582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.492676 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.493167 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.493647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.494991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:32.495900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.498028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.509881 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba4e4a-2bc2-4a21-81ed-bde1d41a3dbd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.531751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 536/1064] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:32.533092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.564451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.573808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.575258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.575767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.576278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.578504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc, user.id=1877b3fdea9d4a44b2179895a87ecf80) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:32.579091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.581838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.592585 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 529/1065] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/role_assignments?user.id=1877b3fdea9d4a44b2179895a87ecf80&scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:32.678352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e665-6330-4814-be55-e6480452cd2e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.729544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.730423 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.730711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.730992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.740063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.741593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:32.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:32.759436 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:32.761780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:32.762413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 537/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:32.766832 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:32.842029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:32.902308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01a413-804f-47b1-a015-84df2404bad7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:32.958281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:32.958964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:32.959244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:32.959787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:32.960646 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:32.960969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:32.962448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eed63b-1f41-44da-8c28-a71141404978 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:32.982353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 530/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:32.988983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.095961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.132120 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03844d3-b9e8-4e10-9f45-6db02db8ba44 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.156302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631fea2-d367-4117-884d-18f75390d08a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.156600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631fea2-d367-4117-884d-18f75390d08a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.157349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631fea2-d367-4117-884d-18f75390d08a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.158464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 531/1068] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:33.163675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.164020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.164215 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.178986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.179798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.180087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.180372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.184810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:33.186470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:33.193481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.195426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:33.215928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:33.216931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:33.219066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:33.219066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:33.219066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:33.219066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:33.220129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 538/1069] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:33.225830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.268530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 532/1070] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:33.291587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.354985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26bcf7-a2ed-4a84-ab4e-132ac51abef7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.357249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.396807 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.397510 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.397833 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.398118 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.398880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:33.399213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.400634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.402467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.422558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 539/1071] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:33.423902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.428194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3807d-b110-4449-a5c0-cf6a633c00d4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.469054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.469904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.470197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.470481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.543872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb6153-6d2b-4512-814a-0781f0e73730 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.548813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.550330 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.587208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.588651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.589053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.589368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.599347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.600819 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.601210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:33.619066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:33.620031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:33.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 540/1072] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:33.627189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.682340 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 533/1073] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:55:33.688920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.699978 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.766499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8bacd-beb8-4e3d-9db0-f2c42056d936 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.815465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.816411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.816834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.817311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.818481 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:33.818974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.820614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.821213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc681a-d606-45fa-b8e7-e988514fce0a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.837357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 541/1074] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:33.845591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.862081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:33.862892 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:33.863187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:33.863468 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:33.874768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:33.876209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:33.878240 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:33.897247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:33.898787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:33.899745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:33.899745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:33.899745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:33.899745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:33.899745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 534/1075] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:33.905653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:33.970227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d1c21-53e4-412c-80a3-66c1685d1995 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:33.986149 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:34.019042 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:34.020344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:34.020629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:34.020916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/a2d2bc3f0a524c48bc9e42c326ea6fc0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:34.023272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=80890a6017284bf090892fbcbb618f7f, role_id=a2d2bc3f0a524c48bc9e42c326ea6fc0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:34.024721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47cfcb-0919-41bf-868b-98ac66d2d41e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:34.031866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2956a-95fe-48f9-9720-20d287cfe992 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:34.042392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 542/1076] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:55:33 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/80890a6017284bf090892fbcbb618f7f/roles/a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:55:34.083906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:34.084869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:34.085188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:34.085452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:34.086210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:34.086518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:34.087928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea46c1-d7d5-49a5-b97c-c3c80aaf406c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:34.102473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 535/1077] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:34.681796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de96e-ed39-448c-97e3-a6f0790a0814 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:34.682335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de96e-ed39-448c-97e3-a6f0790a0814 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:34.682335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de96e-ed39-448c-97e3-a6f0790a0814 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:34.683726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 543/1078] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:34.689887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:34.690310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:34.690452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:34.708059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:34.708713 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:34.760282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:34.778301 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 536/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:34.800957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de96e-ed39-448c-97e3-a6f0790a0814 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:34.906080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de96e-ed39-448c-97e3-a6f0790a0814 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:34.939619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:34.940334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:34.940619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:34.940918 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:34.952244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:34.960144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:34.961349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:34.962980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 544/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:34.967516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:35.068552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fcf3d-abb3-4837-921d-852ecb69a1b9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:35.103688 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:35.104555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:35.104852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:35.105232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:35.106100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:35.106488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:35.108107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:35.117570 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 537/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:35.123557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:35.160690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:35.236102 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7730c10-34f9-4b6f-8252-12c83ab934e4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:35.271697 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:35.272573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:35.272857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:35.273134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] PATH_INFO: `/v3/users/1877b3fdea9d4a44b2179895a87ecf80` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:35.274336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] RBAC: Authorizing `identity:get_user(user_id=1877b3fdea9d4a44b2179895a87ecf80)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:35.275764 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:35.277426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 545/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:35 2022] GET /identity/v3/users/1877b3fdea9d4a44b2179895a87ecf80 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:35.284716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:35.354972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:35.409481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7068250-1ab0-4844-bcff-6357e424380b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:35.443137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:35.444003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:35.444286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:35.444560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:35.445787 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:35.447228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8a684-0a74-48f0-88eb-a3def06930ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:35.448933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 538/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:35 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:35.454857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:35.561111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5212-b878-4b99-8892-06d6ad9fab20 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:35.562864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:35.598434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:35.599321 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:35.599615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:35.599888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc/users/1877b3fdea9d4a44b2179895a87ecf80/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:35.601890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] RBAC: Authorizing `identity:create_grant(project_id=31fd1843a64f4f7ab5dd512f13791bdc, user_id=1877b3fdea9d4a44b2179895a87ecf80, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:35.603344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596184d-e4da-4ad9-8343-0eefb11eb681 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:35.616612 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 546/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:55:35 2022] PUT /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc/users/1877b3fdea9d4a44b2179895a87ecf80/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:36.110103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb369173-714e-4c61-9a5e-d157281d4ea5 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.110638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb369173-714e-4c61-9a5e-d157281d4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.110638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb369173-714e-4c61-9a5e-d157281d4ea5 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.112083 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 539/1085] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:36.117431 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.117910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.118096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.137434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:36.138650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:36.205278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 547/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:36.228848 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb369173-714e-4c61-9a5e-d157281d4ea5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:36.331269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb369173-714e-4c61-9a5e-d157281d4ea5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:36.345409 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:36.372445 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:36.373187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.373478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.373800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.384712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:36.393041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:36.394035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:36.394899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 540/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 11 11:55:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:36.400222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:36.495872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:36.505604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdd6b-df09-41e9-b7c2-f93077bf116d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:36.538626 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:36.539291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.539579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.539857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.540600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:36.540912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:36.542298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:36.550589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 548/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:55:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:36.556237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:36.654032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00746-1c33-4b03-9fb7-794cb144ecff None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:36.692963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:36.693796 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.694087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.694368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.704951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:36.706422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:36.724034 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:36.725149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:36.726127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:36.726127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:36.726127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:36.726127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:36.726127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 541/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:36.731532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:36.841393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7d0b-b1ad-4396-8bc1-661f78cafb3c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:36.877638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:36.878328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:36.878619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:36.878902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:36.879640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:36.879956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:36.881336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:36.895154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 549/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:36.900740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:36.932378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.009082 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729502-dad2-45b1-9daa-4bfd359727b4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.044136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:37.045222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.045222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.045493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.053783 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:37.055401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:55:37.070098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:37.071251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:37.072246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 542/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 11 11:55:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:37.077056 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.094390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.187857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a61ca-4993-4fb0-870b-1bee1a872f5c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.222908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:37.223576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.223862 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.224141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.225346 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:37.225926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:37.228299 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:37.246746 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 550/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:37.252653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.348790 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.365131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938bd95-1ce7-425f-a5d5-719b26dd6d18 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.400914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:37.401568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.401887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.402218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.476193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:37.477580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9defcf-ac3f-4260-8f39-e804bcbc1105 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:37.564160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.580407 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 543/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:37.586648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.655185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.685434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56493-030e-4570-b449-1fdb91daea08 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.720867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:37.721662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.721977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.722254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.732889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:37.734287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128b01-e05c-4da1-afb0-c6d00d4178c9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.734426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:37.734572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128b01-e05c-4da1-afb0-c6d00d4178c9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.734856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128b01-e05c-4da1-afb0-c6d00d4178c9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.736243 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 544/1094] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:37.741298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.741593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.741911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:55:37.753323 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:37.754457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:37.755397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:55:37.755397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:55:37.755397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:55:37.755397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:37.755397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 551/1095] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:37.759385 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:37.760267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.760581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:37.831621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 545/1096] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:55:37.855815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.863401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef78739-aa21-456d-a8c9-5965a8f42f3c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.889179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:37.900578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:37.901247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:37.901539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:37.901851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:37.902593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:37.902905 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:37.904286 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:37.916636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 552/1097] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:37.922813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:37.966924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa860b-61bb-4615-b436-6ecdc73deceb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:37.984953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.003659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.004477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.004756 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.005030 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.015831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:38.023493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:38.025217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:55:38.026956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 546/1098] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:38.029528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7bd93-8fc5-4819-a695-39f4a87e851b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.030732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.066287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.067090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.067373 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.067649 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.074462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.075912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:55:38.090673 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:38.091727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:38.092855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 553/1099] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:55:38.098392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.139967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a445f43-1cd7-4f2b-9748-2bdf2aa41c7d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.147847 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.174664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.175333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.175620 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.175901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.176658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:38.176974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.178392 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.187956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 547/1100] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:38.193755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.205087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92740-9563-4589-886a-2459b13b3274 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.233056 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.244456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.245127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.245414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.245715 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.246513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:38.246827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.248254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.260096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 554/1101] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:38.266226 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.296595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af26f-fd2b-4567-b4ba-07e399a8e5cd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.331560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.332430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.332710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.332982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] PATH_INFO: `/v3/users/1877b3fdea9d4a44b2179895a87ecf80` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.334224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1877b3fdea9d4a44b2179895a87ecf80)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.335916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.337556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 548/1102] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/users/1877b3fdea9d4a44b2179895a87ecf80 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:38.343751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.377181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1765-8737-434a-b4fd-b46acb96fe99 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.379604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.413152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.413839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.414125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.414401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.415225 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] RBAC: Adding query filter params (role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=80890a6017284bf090892fbcbb618f7f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:38.415537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.416960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.423598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 555/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/role_assignments?user.id=80890a6017284bf090892fbcbb618f7f&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=a2d2bc3f0a524c48bc9e42c326ea6fc0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:38.452091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2a4b-db62-49c6-8861-7f130b9797a9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.460489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.494266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.494999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.495219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.495493 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.580804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.582309 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.688725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 549/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:55:38.695133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.811458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d1f0-2426-4c13-a240-9d97537bf77d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:38.856502 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:38.857432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:38.857751 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:38.858038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:38.859408 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:38.860902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7b39f-9dcb-4945-9de0-a5c6e73df269 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:38.862612 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 556/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:55:38.868899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:38.878094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:38.975922 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ede7f-6111-43c2-813a-c29c7defec81 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:39.011553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:39.012219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:39.012505 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:39.012793 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:39.013637 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc, user.id=1877b3fdea9d4a44b2179895a87ecf80) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:39.014022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:39.016114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd141d4-a8e2-4188-ad96-4348c8c8af72 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:39.022448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 550/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:38 2022] GET /identity/v3/role_assignments?user.id=1877b3fdea9d4a44b2179895a87ecf80&scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:41.054213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358de4d-dbce-48b5-9a10-1b90dcc87d6f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:41.054748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358de4d-dbce-48b5-9a10-1b90dcc87d6f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:41.054748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358de4d-dbce-48b5-9a10-1b90dcc87d6f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:41.056145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 557/1107] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:55:41.061383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:41.062428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:41.062521 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:41.078761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:41.079932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:41.144206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:41.149563 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 551/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:41.174525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358de4d-dbce-48b5-9a10-1b90dcc87d6f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:41.211679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:41.279615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358de4d-dbce-48b5-9a10-1b90dcc87d6f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:41.318162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:41.318878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:41.319159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:41.319434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:41.320609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:41.322068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd36313-2f0d-4511-b653-e345e8b8145b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:41.323687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 558/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:41.329755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:41.433168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ed3e5-11e8-4236-bd7e-9538ba6cdb58 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:41.469160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:41.469867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:41.470159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:41.470443 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:41.471168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:41.472605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ae50-615c-42c0-99ba-e57fee9409ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:41.492461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 552/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:55:41 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:43.581019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933799e5-d903-4592-8d21-009f6301fc6c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:43.581577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933799e5-d903-4592-8d21-009f6301fc6c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:43.581577 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933799e5-d903-4592-8d21-009f6301fc6c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:43.583040 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 559/1111] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:43.588373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:43.588772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:43.588927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:43.605150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:43.606351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:43.654909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:43.672024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 553/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:43.695989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933799e5-d903-4592-8d21-009f6301fc6c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:43.795374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933799e5-d903-4592-8d21-009f6301fc6c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:43.798019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:43.829883 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:43.831135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:43.831135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:43.831341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] PATH_INFO: `/v3/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:43.832756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] RBAC: Authorizing `identity:get_role(role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:43.834157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:43.835793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 560/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:43 2022] GET /identity/v3/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:43.841766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:43.941939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad3fda-0d61-4072-b576-ef8e672b583e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:43.974524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:43.975386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:43.975664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:43.975935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] PATH_INFO: `/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:43.989883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] RBAC: Authorizing `identity:get_user(user_id=e5f0a68d4ebe49a9abe9d85f01a2ad8e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:43.991384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:43.993087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 554/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:43 2022] GET /identity/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:43.998875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:44.095256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d095cb-3b4c-4a8c-ba59-85072d5b1ab7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:44.130551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:44.131212 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:44.131501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:44.131780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:44.173468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:44.200341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:44.201830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:44.306744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 561/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:44.312765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:44.361148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:44.418318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f894b0-33b0-4504-9259-a07e4d593ede None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:44.458320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:44.459594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:44.459883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:44.460163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:44.461411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:44.462875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f0c18-f0b7-4fe0-9ee4-49fd91deb392 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:44.464606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 555/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:44 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:44.471230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:44.538262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:44.567198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b187d-e17b-415a-ac9d-0cb7bcb52d8b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:44.600921 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:44.601595 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:44.601868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:44.602146 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:44.602974 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc, user.id=e5f0a68d4ebe49a9abe9d85f01a2ad8e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:44.603289 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:44.604709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ebb28-abcf-468a-9a93-5f97f09304d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:44.610301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 562/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:44 2022] GET /identity/v3/role_assignments?user.id=e5f0a68d4ebe49a9abe9d85f01a2ad8e&scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:46.625524 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec281a-f667-4f74-8cc3-73996f07e956 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:46.626107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec281a-f667-4f74-8cc3-73996f07e956 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:46.626107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec281a-f667-4f74-8cc3-73996f07e956 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:46.627581 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 556/1118] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:46.632786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:46.633172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:46.633344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:46.650497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:46.651679 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:46.720072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 563/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:46.743054 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec281a-f667-4f74-8cc3-73996f07e956 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:46.852718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec281a-f667-4f74-8cc3-73996f07e956 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:46.868410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:46.891407 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:46.892228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:46.892490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:46.892770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] PATH_INFO: `/v3/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:46.894395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] RBAC: Authorizing `identity:get_role(role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:46.895779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:46.897196 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 557/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:46 2022] GET /identity/v3/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:46.903174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:46.971322 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:47.005593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b30f89-bc35-434d-9f70-83b8c4de5315 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:47.039602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:47.040473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:47.040763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:47.041047 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] PATH_INFO: `/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:47.042256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=e5f0a68d4ebe49a9abe9d85f01a2ad8e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:47.043708 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:47.045353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 564/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:46 2022] GET /identity/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:47.051245 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:47.151611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff09ab-19b0-42b3-a6d5-aa831a4bb968 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:47.186293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:47.187836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:47.188123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:47.188402 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:47.190252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:47.192121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae4a1f-48c8-444c-ad6d-58aee91eba7c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:47.193855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 558/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:47 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:47.199927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:47.300161 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8040b-7178-4a88-b721-558679625fa7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:47.333134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:47.334046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:47.334591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:47.334868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:47.337072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] RBAC: Authorizing `identity:create_grant(project_id=31fd1843a64f4f7ab5dd512f13791bdc, user_id=e5f0a68d4ebe49a9abe9d85f01a2ad8e, role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:47.338534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f219e96-42c1-484d-a8c0-1c626c78220c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:47.340720 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:47.352733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 565/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:55:47 2022] PUT /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:49.351410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d58ef-9a98-4fab-b3b9-24e2325b489a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:49.351976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d58ef-9a98-4fab-b3b9-24e2325b489a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:49.351976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d58ef-9a98-4fab-b3b9-24e2325b489a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:49.353332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 559/1124] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:49.358796 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:49.359247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:49.359367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:49.376020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:49.377115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:49.444432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 566/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:49.470631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d58ef-9a98-4fab-b3b9-24e2325b489a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:49.520817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:49.571569 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d58ef-9a98-4fab-b3b9-24e2325b489a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:49.606619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:49.607476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:49.607766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:49.608070 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] PATH_INFO: `/v3/roles/26e3698098ff479ea7207c4cd8acc9b5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:49.609575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] RBAC: Authorizing `identity:get_role(role_id=26e3698098ff479ea7207c4cd8acc9b5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:49.610974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:49.612376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 560/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:49 2022] GET /identity/v3/roles/26e3698098ff479ea7207c4cd8acc9b5 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:49.618341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:49.713163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c473a5-eed2-4dd2-b529-4eb9fd4ef1e0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:49.736302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:49.747966 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:49.749372 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:49.749648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:49.749950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] PATH_INFO: `/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:49.751130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] RBAC: Authorizing `identity:get_user(user_id=e5f0a68d4ebe49a9abe9d85f01a2ad8e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:49.752574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:49.754228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 567/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:49 2022] GET /identity/v3/users/e5f0a68d4ebe49a9abe9d85f01a2ad8e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:49.759990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:49.857133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433657-cc85-4776-adb7-9c67ef40bf72 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:49.874847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:49.893191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:49.893867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:49.894152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:49.894426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:49.966961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:49.968339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:50.072478 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 561/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:50.078557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:50.182470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e884-96ab-412d-b6a8-90ba04d28140 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:50.215788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:50.216646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:50.216932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:50.217254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] PATH_INFO: `/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:50.218920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] RBAC: Authorizing `identity:get_project(project_id=31fd1843a64f4f7ab5dd512f13791bdc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:50.220354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114139-9d25-4231-9077-e44f71cc896d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:50.222094 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 568/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:50 2022] GET /identity/v3/projects/31fd1843a64f4f7ab5dd512f13791bdc => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:50.228360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:50.246173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:50.327339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20c9e-0515-4111-8282-11ec48211805 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:50.361567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:50.362257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:50.362528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:50.362809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:50.363637 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] RBAC: Adding query filter params (role.id=26e3698098ff479ea7207c4cd8acc9b5, scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc, user.id=e5f0a68d4ebe49a9abe9d85f01a2ad8e) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:50.363955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:50.365378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448e46f-e964-40f2-94a0-ddc9c0a9dd11 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:50.371142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 562/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:50 2022] GET /identity/v3/role_assignments?user.id=e5f0a68d4ebe49a9abe9d85f01a2ad8e&scope.project.id=31fd1843a64f4f7ab5dd512f13791bdc&role.id=26e3698098ff479ea7207c4cd8acc9b5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:52.428458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be758-d7a4-44e9-b062-1e67aab165d1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:52.428982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be758-d7a4-44e9-b062-1e67aab165d1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:52.428982 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be758-d7a4-44e9-b062-1e67aab165d1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:52.430374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 569/1131] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:52.437483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:52.437954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:52.438085 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:52.455372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:52.456578 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:52.524764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 563/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:52.551462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be758-d7a4-44e9-b062-1e67aab165d1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:52.614459 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:52.656705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be758-d7a4-44e9-b062-1e67aab165d1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:52.689914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:52.690637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:52.690926 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:52.691205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:52.692388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:52.693868 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339baa6f-3219-48c3-9ffe-5e4b5f29e501 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:52.695501 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 570/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:52.701565 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:52.713953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:52.800470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d1b9d-663c-45ac-97d2-75cf2137765b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:52.833870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:52.834779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:52.835061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:52.835335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:52.836042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:52.837458 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f932fd2-149a-4774-88bc-9806e3bdf8d9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:52.865323 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 564/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:55:52 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:54.951981 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61035e5-3ba6-4fae-ad02-a83f8053585f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:54.952515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61035e5-3ba6-4fae-ad02-a83f8053585f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:54.952515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61035e5-3ba6-4fae-ad02-a83f8053585f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:54.953931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 571/1135] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:54.959755 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:54.960187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:54.960316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:54.977849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:54.979886 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:55.037463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:55.049126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 565/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:55.074192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61035e5-3ba6-4fae-ad02-a83f8053585f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:55.175977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61035e5-3ba6-4fae-ad02-a83f8053585f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:55.201351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:55.210117 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:55.210854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:55.211132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:55.211404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:55.212570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:55.214035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7943fb-e3d9-4515-814e-c7db2ef0b114 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:55.215653 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 572/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:55:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:55.222019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:55.318047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d39e6-b4d2-49b9-8622-e786dbf109a2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:55.352216 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:55.352883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:55.353166 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:55.353446 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:55.354194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:55.355614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4669a-07e5-4d00-bd0a-5d653340a07d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:55.375262 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 566/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:55:55 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:57.462344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6519be4-98c0-4257-a168-27e22b29814a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:57.462855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6519be4-98c0-4257-a168-27e22b29814a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:57.462855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6519be4-98c0-4257-a168-27e22b29814a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:57.464210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 573/1139] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:55:57.469419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:57.469822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:57.470087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:57.486980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:57.488171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:57.531041 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:57.555698 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 567/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:55:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:55:57.579940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6519be4-98c0-4257-a168-27e22b29814a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:57.680202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6519be4-98c0-4257-a168-27e22b29814a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:57.714098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:57.714824 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:57.715116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:57.715410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:57.725287 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:55:57.732949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:55:57.734140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:55:57.735306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 574/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:55:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:55:57.740327 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:57.837491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66398d-707d-4ce6-88d3-c616422c290e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:57.870396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:57.871063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:57.871352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:57.871634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:57.872384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:57.872698 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:57.874611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:57.882828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 568/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:55:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:57.888592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:57.943593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:57.988027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41700f4-7d61-4acf-99af-d4602339e935 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:58.028750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:58.029649 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:58.029968 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:58.030245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] PATH_INFO: `/v3/users/2e0dd5ae450d401681c3517d1df6c6c9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:58.043349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=2e0dd5ae450d401681c3517d1df6c6c9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:58.044817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:58.046579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 575/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:55:57 2022] GET /identity/v3/users/2e0dd5ae450d401681c3517d1df6c6c9 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:58.052380 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:58.094290 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:58.153326 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa92e7c-ed2e-482a-906e-2f2f4cbda60e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:58.185965 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:58.187101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:58.187387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:58.187660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:58.255209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:58.256851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:58.296975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:58.355952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 569/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:55:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:55:58.362645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:58.459898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e16f0-6bde-4b0b-a0f5-45198b84a4e5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:58.474084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:58.494411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:58.495276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:58.495564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:58.495839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] PATH_INFO: `/v3/projects/4f52613ee86148e28397dc9a693341a9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:58.497009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] RBAC: Authorizing `identity:get_project(project_id=4f52613ee86148e28397dc9a693341a9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:58.498523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408721f3-a013-410d-9604-fadbcfd348be None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:58.500225 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 576/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:55:58 2022] GET /identity/v3/projects/4f52613ee86148e28397dc9a693341a9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:58.506421 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:58.608489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e71e28-6185-47c1-a0b9-3203818a6ac7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:58.642061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:58.642706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:58.642986 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:58.643259 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:58.644080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=4f52613ee86148e28397dc9a693341a9, user.id=2e0dd5ae450d401681c3517d1df6c6c9) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:58.646042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:58.646042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7398d6-2513-44b1-82be-c0dc47eaf24e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:58.651258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 570/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:55:58 2022] GET /identity/v3/role_assignments?user.id=2e0dd5ae450d401681c3517d1df6c6c9&scope.project.id=4f52613ee86148e28397dc9a693341a9&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:00.663914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd12fe6-df56-4343-881b-19c244189d98 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:00.664532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd12fe6-df56-4343-881b-19c244189d98 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:00.664532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd12fe6-df56-4343-881b-19c244189d98 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:00.666028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 577/1147] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:00.671982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:00.672403 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:00.672522 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:00.678435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:00.689769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:00.690950 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:00.755775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 571/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:00.778256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd12fe6-df56-4343-881b-19c244189d98 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:00.877604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd12fe6-df56-4343-881b-19c244189d98 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:00.910238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:00.911027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:00.911308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:00.911582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:00.921908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:56:00.929417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:00.930564 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:56:00.931793 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 578/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:56:00.936802 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:01.033777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e55739-15c8-4159-8a7e-06b4d9494d8d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:01.068654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:01.069771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:01.070271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:01.070611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:01.071503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:01.071820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:01.073205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:01.074710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:01.083481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 572/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:01.089623 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:01.135011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:01.192408 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ed63f-19fe-415e-9e88-2b83bb854083 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:01.227272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:01.228155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:01.228440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:01.228719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] PATH_INFO: `/v3/users/2e0dd5ae450d401681c3517d1df6c6c9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:01.229935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] RBAC: Authorizing `identity:get_user(user_id=2e0dd5ae450d401681c3517d1df6c6c9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:01.231356 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:01.233046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 579/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:56:01 2022] GET /identity/v3/users/2e0dd5ae450d401681c3517d1df6c6c9 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:01.240900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:01.348315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06563c6-694d-4d55-9ce5-66e8ad86b0cc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:01.386675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:01.387578 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:01.387867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:01.388150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] PATH_INFO: `/v3/projects/4f52613ee86148e28397dc9a693341a9` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:01.389391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=4f52613ee86148e28397dc9a693341a9)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:01.390872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6c99-753a-45ae-bf7f-725bffa97fa3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:01.392599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 573/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:56:01 2022] GET /identity/v3/projects/4f52613ee86148e28397dc9a693341a9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:01.400044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:01.505579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf12c0-b628-4d13-ac03-5df38a101b1c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:01.515794 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:01.539398 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:01.540295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:01.540585 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:01.540866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] PATH_INFO: `/v3/projects/4f52613ee86148e28397dc9a693341a9/users/2e0dd5ae450d401681c3517d1df6c6c9/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:01.542851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] RBAC: Authorizing `identity:create_grant(project_id=4f52613ee86148e28397dc9a693341a9, user_id=2e0dd5ae450d401681c3517d1df6c6c9, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:01.544294 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c084-4a8f-46d9-8641-6c99b5551c9a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:01.557667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 580/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:56:01 2022] PUT /identity/v3/projects/4f52613ee86148e28397dc9a693341a9/users/2e0dd5ae450d401681c3517d1df6c6c9/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:03.585346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc6d8-2120-4558-b9da-9673abdcc59b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:03.585949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc6d8-2120-4558-b9da-9673abdcc59b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:03.585949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc6d8-2120-4558-b9da-9673abdcc59b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:03.587306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 574/1154] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:03.592781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:03.593268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:03.593361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:03.610632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:03.611822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:03.677463 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 581/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:03.701820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cc6d8-2120-4558-b9da-9673abdcc59b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:03.767813 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:03.816489 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cc6d8-2120-4558-b9da-9673abdcc59b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:03.854168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:03.854895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:03.855182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:03.855462 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:03.867229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:56:03.875237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:03.876464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:56:03.877383 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 575/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:56:03.882694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:03.979867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:03.983431 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e1612-5071-46f8-a203-a0b8ecdeb2ce None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:04.017528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:04.018227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:04.018513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:04.018790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:04.019532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:04.019848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:04.021205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:04.029153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 582/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:04.035621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:04.133754 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eb080-b48a-4117-9f50-8706e0b7d4d2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:04.157332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:04.168120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:04.169001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:04.169288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:04.169574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] PATH_INFO: `/v3/users/2e0dd5ae450d401681c3517d1df6c6c9` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:04.170827 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] RBAC: Authorizing `identity:get_user(user_id=2e0dd5ae450d401681c3517d1df6c6c9)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:04.172260 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:04.173935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 576/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:56:04 2022] GET /identity/v3/users/2e0dd5ae450d401681c3517d1df6c6c9 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:04.179658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:04.277320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079e51-8c0a-4a83-a231-a57451f4664f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:04.311983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:04.312656 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:04.312940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:04.313269 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:04.342645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:04.380593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:04.381999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:04.477079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 583/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:56:04.483077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:04.580506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a034d-a0c1-411f-91f2-62a5429f9472 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:04.614214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:04.615077 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:04.615415 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:04.615701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] PATH_INFO: `/v3/projects/4f52613ee86148e28397dc9a693341a9` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:04.616876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] RBAC: Authorizing `identity:get_project(project_id=4f52613ee86148e28397dc9a693341a9)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:04.618332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf57b94-1b58-4918-9cde-6305bdd0dc02 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:04.620029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 577/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:56:04 2022] GET /identity/v3/projects/4f52613ee86148e28397dc9a693341a9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:04.626340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:04.685951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:04.724534 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cd3f7-8c2f-4702-a677-c165a8e86ab2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:04.757184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:04.757863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:04.758154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:04.758434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:04.759264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=4f52613ee86148e28397dc9a693341a9, user.id=2e0dd5ae450d401681c3517d1df6c6c9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:04.759582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:04.761002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2b2a0-50c0-49bf-950f-cf7eb7fd622a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:04.766839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 584/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:56:04 2022] GET /identity/v3/role_assignments?user.id=2e0dd5ae450d401681c3517d1df6c6c9&scope.project.id=4f52613ee86148e28397dc9a693341a9&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:06.799673 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adf05f-b05b-4ea3-8d8f-204d2e977143 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:06.800238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adf05f-b05b-4ea3-8d8f-204d2e977143 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:06.800238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adf05f-b05b-4ea3-8d8f-204d2e977143 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:06.801625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 578/1162] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:06.806978 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:06.807395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:06.807544 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:06.824298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:06.825471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:06.891424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 585/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:06.914696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adf05f-b05b-4ea3-8d8f-204d2e977143 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:06.973900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:07.015540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adf05f-b05b-4ea3-8d8f-204d2e977143 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:07.047898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:07.048537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:07.048823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:07.049101 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:07.115529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:07.116909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:07.176154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:07.217233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 579/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:56:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:56:07.223619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:07.277393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:07.320505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81a8f9-7aab-4ac9-8288-0a8bc240fe17 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:07.354886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07d500-e482-4598-8269-321610f418fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:07.355688 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07d500-e482-4598-8269-321610f418fb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:07.355976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07d500-e482-4598-8269-321610f418fb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:07.356254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07d500-e482-4598-8269-321610f418fb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:07.363035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07d500-e482-4598-8269-321610f418fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:07.365255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07d500-e482-4598-8269-321610f418fb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ee07d500-e482-4598-8269-321610f418fb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:07.381444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:07.382714 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:56:07.383606 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 586/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:56:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:56:07.389285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:07.486903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2fdd-4701-4dc6-8049-08e544c28ee4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:07.520157 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:07.520808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:07.521092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:07.521369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:07.522111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:07.522422 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:07.523780 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbf32c-8edb-4d36-8156-f0746f0e91f1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:07.530780 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 580/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 11 11:56:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:09.557494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668035-ebcb-427c-8c8c-8636f9e8d835 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:09.558116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668035-ebcb-427c-8c8c-8636f9e8d835 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:09.558116 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668035-ebcb-427c-8c8c-8636f9e8d835 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:09.559526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 587/1167] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:09.566013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa00f22-d932-4e8b-b6de-136f12b92f80 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:09.566559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa00f22-d932-4e8b-b6de-136f12b92f80 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:09.566559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa00f22-d932-4e8b-b6de-136f12b92f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:09.583942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2fa00f22-d932-4e8b-b6de-136f12b92f80 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:09.585452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2fa00f22-d932-4e8b-b6de-136f12b92f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:09.604193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:09.658272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 581/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:09.680864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03668035-ebcb-427c-8c8c-8636f9e8d835 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:09.781539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03668035-ebcb-427c-8c8c-8636f9e8d835 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:09.797483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:09.815449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:09.816050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:09.816336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:09.816613 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:09.817350 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:09.818747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062e84c-bed1-4168-b37d-668e63c3027b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:09.833152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 588/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Jul 11 11:56:09 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:11.877001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841cde-2a0a-4ff4-8e1b-9eb2d6121aac None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:11.877544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841cde-2a0a-4ff4-8e1b-9eb2d6121aac None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:11.877544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841cde-2a0a-4ff4-8e1b-9eb2d6121aac None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:11.878961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 582/1170] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:11.884198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:11.884618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:11.884787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:11.901912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:11.903108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:11.969101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 589/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:11.992204 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05841cde-2a0a-4ff4-8e1b-9eb2d6121aac None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:12.095315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05841cde-2a0a-4ff4-8e1b-9eb2d6121aac None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:12.130088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:12.130906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:12.131193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:12.131471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] PATH_INFO: `/v3/domains/fff271c9f5ec4996a7b3a3a3b1dea023` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:12.142951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=fff271c9f5ec4996a7b3a3a3b1dea023)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:12.144405 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b8697-6336-4ef1-93a2-81cd27571d8a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:12.146117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 583/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 11 11:56:11 2022] GET /identity/v3/domains/fff271c9f5ec4996a7b3a3a3b1dea023 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:12.152076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:12.250099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33ccb-367c-42c5-9f5f-964910c91b08 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:12.257465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:12.285569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:12.286263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:12.286557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:12.286839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:12.287551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:12.288995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f5bf1-5c6d-4c0f-8163-97a8a40ffb95 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:12.318830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 590/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:56:12 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:14.422197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3f38-592e-4b51-b2c1-6ae04e6d3120 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:14.422722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3f38-592e-4b51-b2c1-6ae04e6d3120 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:14.422722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3f38-592e-4b51-b2c1-6ae04e6d3120 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:14.424062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 584/1174] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:14.429518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:14.430008 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:14.430125 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:14.447233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:14.448427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:14.514813 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 591/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:14.539658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3f38-592e-4b51-b2c1-6ae04e6d3120 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:14.555850 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:14.642624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3f38-592e-4b51-b2c1-6ae04e6d3120 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:14.676369 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:14.677133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:14.677417 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:14.677722 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] PATH_INFO: `/v3/domains/fff271c9f5ec4996a7b3a3a3b1dea023` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:14.678911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fff271c9f5ec4996a7b3a3a3b1dea023)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:14.680337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad21a3c-ab39-455e-89be-3f9af29a6f00 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:14.681993 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 585/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 11 11:56:14 2022] GET /identity/v3/domains/fff271c9f5ec4996a7b3a3a3b1dea023 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:14.688394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:14.784484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:14.797439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71976927-96d1-47ee-a936-e3931bf3f75a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:14.831491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:14.832157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:14.832442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:14.832718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:14.833440 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:14.835169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d593431-fa88-465d-bf38-9cfc30e0a061 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:14.856383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 592/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:56:14 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:16.942900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4436-1700-4778-b06e-d690961a0af6 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:16.943476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4436-1700-4778-b06e-d690961a0af6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:16.943476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4436-1700-4778-b06e-d690961a0af6 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:16.944822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 586/1178] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:16.950232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:16.950667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:16.950767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:16.967463 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:16.968634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:17.032999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 593/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:17.057160 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b4436-1700-4778-b06e-d690961a0af6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:17.140761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:17.161356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b4436-1700-4778-b06e-d690961a0af6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:17.198200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:17.198914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:17.199202 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:17.199482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:17.209822 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:56:17.218048 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:17.219201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:56:17.220382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 587/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:56:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:56:17.225326 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:17.300530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:17.326917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a2f8a-d2cf-4931-867e-1a5630a430eb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:17.360698 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:17.361369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:17.361657 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:17.361986 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:17.362790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:17.363106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:17.364473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:17.372405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 594/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:56:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:17.378361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:17.477628 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31207c99-1afe-407f-a565-c69f2cc9777d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:17.511022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:17.511901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:17.512187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:17.512465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] PATH_INFO: `/v3/users/a2acef67db174c98aab778211b80a4af` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:17.524488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:17.527394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] RBAC: Authorizing `identity:get_user(user_id=a2acef67db174c98aab778211b80a4af)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:17.529900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:17.531970 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 588/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:56:17 2022] GET /identity/v3/users/a2acef67db174c98aab778211b80a4af => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:17.537589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:17.642055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e65d-fb7a-482f-aa69-8315afcee6d8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:17.675293 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:17.675945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:17.676236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:17.676516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:17.679994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:17.744573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:17.746054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:17.845875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 595/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:56:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:56:17.851732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:17.956015 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80085a0f-3b04-4c67-8884-50fd4dfe1e32 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:17.989175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:17.990060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:17.990348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:17.990626 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] PATH_INFO: `/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:17.991807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] RBAC: Authorizing `identity:get_project(project_id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:17.993246 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c5d9e-3f22-4114-beb1-3326a722ff98 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:17.995062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 589/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:56:17 2022] GET /identity/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:18.001624 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:18.035076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:18.099316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772414-37c2-4f26-9ba5-f58afa817793 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:18.132903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:18.133563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:18.133874 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:18.134152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:18.134984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb, user.id=a2acef67db174c98aab778211b80a4af) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:18.135292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:18.136702 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af939-ebdf-4893-bdb6-ccfc778f1da3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:18.142967 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 596/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:56:18 2022] GET /identity/v3/role_assignments?user.id=a2acef67db174c98aab778211b80a4af&scope.project.id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:20.220737 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee576-047f-466e-827d-6d6ec36e1a59 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.221273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee576-047f-466e-827d-6d6ec36e1a59 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.221273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee576-047f-466e-827d-6d6ec36e1a59 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.222803 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 590/1186] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:20.228354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.228825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.228943 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.246127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:20.247401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:20.319523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 597/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:20.344792 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ee576-047f-466e-827d-6d6ec36e1a59 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:20.417759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:20.457556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ee576-047f-466e-827d-6d6ec36e1a59 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:20.494333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:20.495148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.495424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.495697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.506964 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:56:20.515283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:20.516461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:56:20.517447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 591/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:56:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:56:20.522941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:20.552961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:20.627184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303c15c-d246-46d5-b033-3a01aaec0f83 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:20.662328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:20.663003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.663278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.663549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.664307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:20.664614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:20.665999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:20.675385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 598/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:56:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:20.681657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:20.787905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5febd5-cb9b-4bec-9af5-5185e787c314 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:20.815334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:20.823496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:20.824376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.824656 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.824933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] PATH_INFO: `/v3/users/a2acef67db174c98aab778211b80a4af` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.826162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] RBAC: Authorizing `identity:get_user(user_id=a2acef67db174c98aab778211b80a4af)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:20.827590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:20.829319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 592/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:56:20 2022] GET /identity/v3/users/a2acef67db174c98aab778211b80a4af => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:20.843253 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:20.940444 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:20.950452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ec7ce-8bfa-48e6-b27d-50a5e0eca7a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:20.990508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:20.991476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:20.991895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:20.992241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] PATH_INFO: `/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:20.993616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] RBAC: Authorizing `identity:get_project(project_id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:20.995095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a31ec-ab46-4ff5-9559-66990a76d530 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:20.996835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 599/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:56:20 2022] GET /identity/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:21.002980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:21.111989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec1ac-dcf3-463f-af68-cefa91993276 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:21.146799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:21.147715 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:21.147999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:21.148269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] PATH_INFO: `/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb/users/a2acef67db174c98aab778211b80a4af/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:21.150391 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] RBAC: Authorizing `identity:create_grant(project_id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb, user_id=a2acef67db174c98aab778211b80a4af, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:21.151990 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a62c9-16b3-40a9-b679-b8314917f64a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:21.167887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 593/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:56:21 2022] PUT /identity/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb/users/a2acef67db174c98aab778211b80a4af/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:23.209960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455532-b5b6-4c93-bfe0-caee3ffcc74d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.210536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455532-b5b6-4c93-bfe0-caee3ffcc74d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.210536 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455532-b5b6-4c93-bfe0-caee3ffcc74d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:23.211941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 600/1193] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:56:23.218156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.218687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.218793 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:23.236463 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:23.237718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:23.293977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:23.312739 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 594/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:56:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:56:23.338087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2455532-b5b6-4c93-bfe0-caee3ffcc74d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:23.439063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2455532-b5b6-4c93-bfe0-caee3ffcc74d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:23.471990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:23.472788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.473070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.473343 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:23.484095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:56:23.491851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:56:23.492987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:56:23.494097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 601/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:56:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:56:23.499766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:23.596337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162502a2-0ab5-4392-a1d2-e6957861d919 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:23.634528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:23.635199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.635486 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.635771 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:23.636518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:23.636834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:23.638247 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:23.646814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 595/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:56:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:23.652744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:23.674725 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:23.755064 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750c31-7814-494d-895f-bef6216e2c55 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:23.789220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c851c-b3df-4096-b487-897938d93327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:23.790127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c851c-b3df-4096-b487-897938d93327 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.790410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c851c-b3df-4096-b487-897938d93327 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.790685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c851c-b3df-4096-b487-897938d93327 None admin] PATH_INFO: `/v3/users/a2acef67db174c98aab778211b80a4af` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:23.791894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c851c-b3df-4096-b487-897938d93327 None admin] RBAC: Authorizing `identity:get_user(user_id=a2acef67db174c98aab778211b80a4af)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:23.793442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c851c-b3df-4096-b487-897938d93327 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:23.796175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 602/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 11 11:56:23 2022] GET /identity/v3/users/a2acef67db174c98aab778211b80a4af => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:23.802998 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:23.857739 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:23.924607 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370b68-d18c-4d47-8bd8-8b10117faf3b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:23.966657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:23.967345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:23.967634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:23.967915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:24.040860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:24.042250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:24.091066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:24.144084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 596/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:56:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:56:24.150067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c851c-b3df-4096-b487-897938d93327 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:24.237402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:24.249812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c851c-b3df-4096-b487-897938d93327 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:24.294357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:24.295718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:24.295998 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:24.296307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] PATH_INFO: `/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:24.297558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] RBAC: Authorizing `identity:get_project(project_id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:24.299038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4aabad-88ba-42d6-a227-c2fb486a1eaa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:24.300774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 603/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 11 11:56:24 2022] GET /identity/v3/projects/5ccba788ee8c49ab9c8cd4ef1ac5eeeb => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:24.307103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:24.418644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad745ae-d2f2-4779-845d-214de274c08b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:24.461256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:24.462313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:24.462775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:24.463257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:24.465177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb, user.id=a2acef67db174c98aab778211b80a4af) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:24.465642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:24.468502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851fb10-1f99-41e0-a9c4-ecf33f68466d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:24.475808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 597/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:56:24 2022] GET /identity/v3/role_assignments?user.id=a2acef67db174c98aab778211b80a4af&scope.project.id=5ccba788ee8c49ab9c8cd4ef1ac5eeeb&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:34.289654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=22 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:34.472568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=94 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:22.029363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec408-5529-4a99-95cf-3c63cc4c1d76 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:22.030235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec408-5529-4a99-95cf-3c63cc4c1d76 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:22.030235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec408-5529-4a99-95cf-3c63cc4c1d76 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:22.031958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 604/1201] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:22.037810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:22.038372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:22.038372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:22.077337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:22.079766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:22.205051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 598/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:22.232242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acec408-5529-4a99-95cf-3c63cc4c1d76 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:22.377703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acec408-5529-4a99-95cf-3c63cc4c1d76 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:22.427591 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:22.429114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:22.429457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:22.429985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:22.445484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:22.447628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:22.474950 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:22.476052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:22.477127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:22.477127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:22.477127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:22.477127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:22.478897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 605/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:22.485741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:22.635684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba43c02-17bc-4d95-9588-28025544f67f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:22.684900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:22.692470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:22.693226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:22.693560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:22.693893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:22.694553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:22.694862 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:22.696297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0feff10-ed3b-4887-822a-5077c77352b4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:22.711348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 599/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:22.717240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:22.786121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:22.873785 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e63c7-87fd-42eb-89d9-93355d2fa997 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:22.943799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:22.944962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:22.944962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:22.945235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:22.946031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:22.947747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2abca-418e-4afb-ace4-e45f1adb42be None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:22.978168 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 606/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 11 11:57:22 2022] POST /identity/v3/users => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:57:25.500889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258584e1-5bde-47ad-b8c0-669070c81e6b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:25.501781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258584e1-5bde-47ad-b8c0-669070c81e6b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:25.501781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258584e1-5bde-47ad-b8c0-669070c81e6b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:25.503696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 600/1206] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:25.512017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:25.513114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:25.513114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:25.537381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:25.539916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:25.645294 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 607/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:25.671909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258584e1-5bde-47ad-b8c0-669070c81e6b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:25.841615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258584e1-5bde-47ad-b8c0-669070c81e6b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:25.894310 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:25.895548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:25.895974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:25.896595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:25.916211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:25.928122 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:25.929325 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:25.930316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:25.937936 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 601/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:57:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:25.943748 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:26.012867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:26.090241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2681d-a931-4d11-8fe1-509ef4dd6d22 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:26.145410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:26.146539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:26.146952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:26.147613 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:26.149249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:26.149661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:26.152948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:26.162955 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 608/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:57:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:26.168894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:26.243519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:26.308999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b56c8-ae7d-494e-a171-c48891311375 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:26.358390 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:26.359534 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:26.359895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:26.360345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:26.376154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:26.377612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:26.397647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:26.399027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:26.400091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:26.400091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:26.400091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:26.400091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:26.400091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 602/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:26.408542 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:26.550940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:26.557225 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c1c1-38f1-4897-8c76-4bb0da49dc03 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:26.628352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:26.629247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:26.629630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:26.629991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:26.630999 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:26.631410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:26.633399 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:26.650473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 609/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:26.656389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:26.814438 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc05c6-163d-4a17-b6d4-a004fbfdcf34 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:26.816164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:26.875602 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:26.876595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:26.876896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:26.877248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:26.889532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:26.892133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:26.915509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:26.916854 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:26.918450 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 603/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:26.927166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:27.113741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66691213-b0db-4a94-8536-ef0f18c6dfc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:27.165786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:27.166395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:27.167032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:27.167032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:27.168347 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:27.168660 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:27.170227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:27.201712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 610/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:27.204071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:27.382947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912da641-e354-4ad0-b287-907ebb1926a0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:27.435147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:27.435873 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:27.436256 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:27.436709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:27.477920 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:27.552337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:27.555211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:27.725963 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 604/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:57:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:57:27.735397 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:27.769571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:27.930701 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e0108-fe10-4e1a-ac59-863b5db7dfc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:27.981174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:27.981889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:27.982237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:27.982566 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:27.999234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:28.001138 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:28.026365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:28.027486 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:28.028914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:28.028914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:28.028914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:28.028914 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:28.031456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 611/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:28.038497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:28.146710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:28.200814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e4206-3def-422d-a10f-8c4a74520c4a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:28.256579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:28.257377 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:28.257660 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:28.257996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:28.258707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:28.259062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:28.260612 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:28.279183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 605/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:28.281165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:28.379098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:28.421889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40edce-bb88-4806-a833-ac1126f0ec54 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:28.481349 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:28.482217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:28.482507 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:28.482786 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:28.491949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:28.493567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:28.510805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:28.511906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:28.512951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:28.516878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 612/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:28.518439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:28.698285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507cdeb-83c5-4900-b80b-efcf4eff130d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:28.740334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:28.760232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:28.761125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:28.761461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:28.761781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:28.763084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:28.763576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:28.765914 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41466fdc-732f-48d1-8cef-c94ff550ce4f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:28.793717 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 606/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:28.801410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.002090 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a3e7-98fa-45fd-b890-4c7f52959de3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.046520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:29.056003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.056911 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.057311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.061153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.061153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=a6db07035b4d472ebc58509502b666ba) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:29.061153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.061153 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7691b23-5f55-4d12-8f81-d0be6a971e4d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:29.067032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 613/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:57:28 2022] GET /identity/v3/role_assignments?user.id=a6db07035b4d472ebc58509502b666ba&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:31.292647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fdf53-d8f6-4690-b5da-9d23b5d7ff81 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:31.293296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fdf53-d8f6-4690-b5da-9d23b5d7ff81 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:31.293296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fdf53-d8f6-4690-b5da-9d23b5d7ff81 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:31.294625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 607/1220] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:31.301156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:31.301794 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:31.301794 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:31.327908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:31.329779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:31.422901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 614/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:31.456779 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fdf53-d8f6-4690-b5da-9d23b5d7ff81 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:31.586155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fdf53-d8f6-4690-b5da-9d23b5d7ff81 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:31.640382 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:31.641532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:31.642076 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:31.642554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:31.665361 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:31.675590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:31.677928 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:31.679477 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 608/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:57:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:31.685935 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:31.800341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b962d-6a6d-46ab-bff9-8020b13fc1dc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:31.831367 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:31.847538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:31.848217 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:31.848627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:31.848930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:31.849927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:31.850275 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:31.851713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:31.861835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 615/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:57:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:31.867889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:31.922365 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:32.000324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246aec0-c34a-4ab4-a71b-d9c862a7c09a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:32.047496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:32.048743 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:32.049179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:32.049655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:32.068510 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:32.070775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:32.095002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:32.097282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:32.099789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:32.099789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:32.099789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:32.099789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:32.099789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 609/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:32.102880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:32.244445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94e53-0f24-4082-8b63-0dabc5807d7a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:32.285973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:32.286520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:32.286810 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:32.287093 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:32.287831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:32.288150 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:32.289535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:32.304557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 616/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:32.311633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:32.429795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:32.439345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac7ba4-5248-4d68-8860-da706f2ef1d8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:32.479218 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:32.481040 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:32.481321 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:32.481591 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:32.489896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:32.491474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:32.507903 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:32.508946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:32.510207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 610/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:32.517175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:32.588925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:32.650583 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364312d7-c8f4-4229-b3ff-ac7eed5f969d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:32.689192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:32.689917 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:32.690181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:32.690455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:32.691302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:32.691651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:32.693304 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:32.721103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 617/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:32.727180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:32.861282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940becf4-6ed3-45eb-9031-2f0c29ff2b49 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:32.908907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:32.909780 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:32.910057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:32.910333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:32.926433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:32.928237 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:32.948490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:32.949955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:32.952192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:32.952192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:32.952192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:32.952192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:32.952192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 611/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:32.957224 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:33.083969 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25163c-dc78-429a-99e4-c96b6bc3c8eb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:33.087312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:33.135616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:33.136454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:33.136851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:33.137309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:33.138380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:33.138839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:33.140906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:33.158009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 618/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:33.167795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:33.182058 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:33.315852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576318e3-5381-4223-ae3f-ff95162bfd28 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:33.359217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:33.360494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:33.360926 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:33.361368 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:33.371227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:33.374088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:33.398837 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:33.400176 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:33.401314 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 612/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:33.408465 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:33.554989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f51f-04c5-42f5-b5d9-73b5d5c66638 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:33.605014 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:33.605727 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:33.606018 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:33.606343 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:33.607137 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:33.607518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:33.609279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69190fe3-e437-4189-9862-afcc29a2cf6f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:33.626642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 619/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:33.633763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:33.725693 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:33.771874 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771338-36d9-49b1-9118-26aecb74e3df None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:33.814839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:33.815804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:33.816131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:33.816753 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/a6db07035b4d472ebc58509502b666ba/roles/65d3647a885247948f9f81b0230ac80e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:33.836933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=a6db07035b4d472ebc58509502b666ba, role_id=65d3647a885247948f9f81b0230ac80e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:33.838559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058eef14-7dbf-45b8-9a76-8198de8efa8c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:33.855953 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 613/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:57:33 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/a6db07035b4d472ebc58509502b666ba/roles/65d3647a885247948f9f81b0230ac80e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:57:36.214932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78a21-c4b9-4438-8043-247cb6b00ded None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:36.215492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78a21-c4b9-4438-8043-247cb6b00ded None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:36.215492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78a21-c4b9-4438-8043-247cb6b00ded None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:36.217247 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 620/1233] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:36.223343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:36.223924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:36.224153 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:36.244795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:36.246330 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:36.331580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 614/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:57:36.356515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c78a21-c4b9-4438-8043-247cb6b00ded None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:36.386637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:36.473619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c78a21-c4b9-4438-8043-247cb6b00ded None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:36.512324 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:36.513268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:36.513628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:36.514003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:36.529455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:36.537547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:36.538772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:57:36.539835 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:36.544084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 621/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 11 11:57:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:36.545360 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:36.601587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:36.664432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999507b-ffc9-485a-9671-3f44930f64e6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:36.719573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:36.720601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:36.721006 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:36.721333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:36.722274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:36.722641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:36.724302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:36.737332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 615/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 11 11:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:36.744596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:36.864134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:36.892024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776d08-ffd6-43b7-8651-720dc269c57b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:36.933988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:36.934724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:36.935011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:36.935293 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:36.946366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:36.947850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:36.967844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:36.969007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:36.969991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:36.969991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:36.969991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:36.969991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:36.969991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 622/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:36.974905 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:37.097084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bdb1e-1ea6-4075-8e8a-8b49a031f9bf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:37.108226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:37.141967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:37.142699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:37.143033 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:37.143332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:37.144102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:37.144451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:37.145846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:37.158580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 616/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:37.165854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:37.279582 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e174-0cd5-4e43-b57b-deaa989f4a6d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:37.316366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:37.317175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:37.317460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:37.317768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:37.320984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:37.326814 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:37.328460 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:37.344291 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:37.345508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:37.352264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 623/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:37.352896 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:37.467658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725138a9-6c4c-42c2-a8c6-4fd67df6afcf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:37.505090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:37.505877 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:37.506224 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:37.506506 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:37.507388 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:37.507702 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:37.509271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:37.531399 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 617/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:37.537415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:37.652337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc843b6-6090-4fc5-b52f-c67cbef0eabe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:37.694552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:37.695314 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:37.695645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:37.695984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:37.767297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:37.775363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:37.777485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:37.915972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 624/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:57:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:57:37.922435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:37.954897 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:38.041989 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88957297-93a8-4e90-848f-fd4e53363472 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:38.084479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:38.085546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:38.086045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:38.086318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:38.097558 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:38.099069 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:38.121768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:38.123724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:38.125132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:38.125132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:38.125132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:38.125132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:38.125132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 618/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:38.129143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:38.243057 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:38.250645 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fba58-e537-4d19-8215-a18ea203a5aa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:38.296045 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:38.296978 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:38.297095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:38.297336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:38.298104 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:38.298423 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:38.300206 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:38.314038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 625/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:38.319828 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:38.412745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:38.440338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005bc5-c33d-4516-b1ff-4e9b9b2358ff None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:38.487103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:38.488371 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:38.488883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:38.489274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:38.497795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:38.499891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:38.522641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:38.523625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:38.524537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:38.524978 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 619/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:38.531487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:38.647062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cedcf-d7ef-4496-ad94-aa38db3696a6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:38.689538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:38.690513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:38.690963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:38.691408 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:38.692441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:38.692903 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:38.694844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723eea4-9af1-403e-8bcd-0eb3bb68930f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:38.710769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 626/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:38.718494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:38.855893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9ee83-1952-4f30-af30-5177c6875484 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:38.911276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:38.911921 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:38.912153 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:38.912432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:38.913370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] RBAC: Adding query filter params (role.id=65d3647a885247948f9f81b0230ac80e, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=a6db07035b4d472ebc58509502b666ba) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:38.913731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:38.915254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0d453-51e2-447f-a57f-9deab6e830ac None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:38.922068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 620/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:57:38 2022] GET /identity/v3/role_assignments?user.id=a6db07035b4d472ebc58509502b666ba&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=65d3647a885247948f9f81b0230ac80e => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:41.328384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4ed1-bad4-40d0-87bd-156099997750 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:41.328952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4ed1-bad4-40d0-87bd-156099997750 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:41.328952 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4ed1-bad4-40d0-87bd-156099997750 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:41.330506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 627/1247] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:41.337494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:41.338080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:41.338080 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:41.343528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:41.357542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:41.359029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:41.440010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 621/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:41.468607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4ed1-bad4-40d0-87bd-156099997750 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:41.487232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:41.584208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4ed1-bad4-40d0-87bd-156099997750 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:41.625002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:41.625889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:41.626113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:41.626426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:41.639667 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:41.649781 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:41.650808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:41.651842 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 628/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:41.657981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:41.773773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef2bad-ccb7-4dba-ba02-8a5b68386b6b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:41.805553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:41.812326 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:41.813295 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:41.813692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:41.813882 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:41.814583 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:41.814902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:41.816398 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:41.827690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 622/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:41.834721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:41.929005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:41.956955 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ea3a-44cc-4643-9295-7a88e785c123 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:41.993769 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:41.994618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:41.994906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:41.995188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:42.007109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:42.008584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:42.027515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:42.029878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:42.030803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:42.030803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:42.030803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:42.030803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:42.030803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 629/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:42.034595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:42.149302 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fe09c-3be7-4d1c-8527-fe589d0bb56a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:42.193146 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9851-c800-4390-99d8-3028f188addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:42.193836 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9851-c800-4390-99d8-3028f188addd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:42.194121 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9851-c800-4390-99d8-3028f188addd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:42.194401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9851-c800-4390-99d8-3028f188addd None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:42.195131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b9851-c800-4390-99d8-3028f188addd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:42.195439 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b9851-c800-4390-99d8-3028f188addd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:42.196825 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b9851-c800-4390-99d8-3028f188addd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:42.209144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 623/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:42.215505 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:42.336618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41b09-0d3f-448c-a36f-16d96b47d9d7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:42.365191 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:42.373879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:42.374803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:42.375088 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:42.375407 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:42.384119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:42.386555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:42.405202 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:42.410053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:42.411415 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 630/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:57:42.412189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9851-c800-4390-99d8-3028f188addd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:42.475637 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:42.531452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9851-c800-4390-99d8-3028f188addd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:42.573432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:42.574214 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:42.574501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:42.574780 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:42.575631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:42.575985 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:42.577434 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:42.598078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 624/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:42.605264 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:42.766433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d0f8-38bc-409f-8eea-26b91e90ba7d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:42.806175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:42.806837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:42.807709 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:42.808259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:42.904121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:42.930997 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:42.935216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:43.063787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 631/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:57:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:57:43.069413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:43.181424 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099e2a-239e-4b13-896d-6e6ac587323f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:43.196638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:43.222242 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:43.223120 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:43.223406 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:43.223690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:43.236488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:43.238010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:43.267575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:43.269394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:43.272042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:43.272042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:43.272042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:43.272042 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:43.275649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 625/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:43.277017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:43.389750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:43.403338 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33de6c-bcf6-4193-9282-7fb83e622039 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:43.442044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:43.443250 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:43.443540 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:43.443821 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:43.444736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:43.445375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:43.447080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:43.459814 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 632/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:43.467183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:43.582627 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba79c08-c0a5-4647-9ebe-57ca01c496f5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:43.636812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:43.637710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:43.638088 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:43.638427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:43.646480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:43.648117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:43.665730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:43.668087 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:43.670057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 626/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:43.674826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:43.800729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933f43-59ed-45b9-b87c-70f67a2a2c45 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:43.849410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:43.850406 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:43.850557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:43.850919 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:43.851733 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:43.852152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:43.854361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a60bf-ea56-470d-87ef-9796e6ddb77f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:43.878749 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:43.879059 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 633/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:43.905283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:43.987279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd4714-e94a-490a-981b-66ab4806a9c7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:44.022320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:44.022994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:44.023280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:44.023560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:44.024395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=a6db07035b4d472ebc58509502b666ba) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:44.024711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:44.026248 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7f4bf-1836-45d0-9654-9e7539395410 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:44.032145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 627/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:57:43 2022] GET /identity/v3/role_assignments?user.id=a6db07035b4d472ebc58509502b666ba&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:46.091584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ba9a-8406-4ffd-b8a5-885ef253e813 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.092179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ba9a-8406-4ffd-b8a5-885ef253e813 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.092179 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ba9a-8406-4ffd-b8a5-885ef253e813 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.093525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 634/1261] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:46.098842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.099298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.099485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.117348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:46.118552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:46.186935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 628/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:57:46.209230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ba9a-8406-4ffd-b8a5-885ef253e813 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:46.224082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:46.318279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ba9a-8406-4ffd-b8a5-885ef253e813 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:46.350374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:46.351089 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.351371 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.351647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.361762 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:46.369396 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:46.371092 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:46.372060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 635/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:57:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:46.377172 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:46.394919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:46.472724 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ce3b6-22ec-4a21-9d8b-28fb038dc2a9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:46.505339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:46.506027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.506313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.506589 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.507331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:46.507639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:46.508999 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:46.516831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 629/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:57:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:46.522506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:46.586936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:46.630987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b9a0e-fd3b-47f1-a7b2-ebf115885e67 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:46.663857 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:46.664654 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.664944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.665227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.676438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:46.677908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:46.694680 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:46.695601 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:46.696555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:46.696555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:46.696555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:46.696555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:46.696555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 636/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:46.701820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:46.781423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:46.800416 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adaddb-237f-4101-9cd0-7c8f8255b4c5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:46.837577 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:46.838984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:46.839357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:46.839729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:46.840638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:46.840979 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:46.842638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:46.856163 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 630/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:46.862845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:46.967507 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202273c-c763-403f-bfe4-5502a3f1af3b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:46.986497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:47.000703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.001514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.001839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.002119 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.009470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.011521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:47.026278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:47.027420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:47.028562 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 637/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:47.033456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.137090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356dd5a-aee2-4357-9ce1-971027e00668 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:47.171312 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.171987 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.172272 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.172551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.173386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:47.173720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.175143 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.184443 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:47.193769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 631/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:47.200375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.299331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c742c-7516-41eb-98ad-eb44b191f2cf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:47.334255 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.335051 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.335337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.335616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.354719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.356190 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:47.373809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:47.374867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:47.375843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:47.375843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:47.375843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:47.375843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:47.375843 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 638/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:47.380908 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.478322 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940296e-f1d8-4bd8-857a-53b26d34bb20 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:47.512414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.513078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.513364 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.513643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.514395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:47.514703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.516072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.527236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 632/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:47.532844 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.564174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:47.633549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18406ec-2ee7-4bad-a7ee-237a3cabd55c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:47.669198 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.670048 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.670337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.670615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.678246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.679707 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:47.696028 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:47.697199 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:47.698205 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 639/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:47.703331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.768496 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:47.812078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ed335-907c-4ad1-9d2f-0c977a1a0fba None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:47.855949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:47.856527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:47.856815 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:47.857092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:47.857906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:47.858221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:47.859647 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28fc75-a5f5-4f22-afb4-b646c72cf93d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:47.875117 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 633/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:47.883128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:47.988002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:47.997007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79ca4-edfa-42f3-8fc2-1fdd470c2aad None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:48.029991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:48.030877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:48.031164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:48.031446 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] PATH_INFO: `/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/a6db07035b4d472ebc58509502b666ba/roles/40fbb4c110e445ea922bbfb223d8701c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:48.033434 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] RBAC: Authorizing `identity:create_grant(project_id=c2af7f192c8d41dfac1733b0da80cd73, user_id=a6db07035b4d472ebc58509502b666ba, role_id=40fbb4c110e445ea922bbfb223d8701c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:48.034897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c037ec2-b676-4e56-b681-15f3e582cafd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:48.048164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 640/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 11 11:57:47 2022] PUT /identity/v3/projects/c2af7f192c8d41dfac1733b0da80cd73/users/a6db07035b4d472ebc58509502b666ba/roles/40fbb4c110e445ea922bbfb223d8701c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:57:50.044824 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410453-bea3-474c-88ee-4dbe04cdd94f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.045414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410453-bea3-474c-88ee-4dbe04cdd94f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.045414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410453-bea3-474c-88ee-4dbe04cdd94f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.046826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 634/1274] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:50.052506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.053049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.053049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.070107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:50.071316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:50.139252 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 641/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:50.163280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91410453-bea3-474c-88ee-4dbe04cdd94f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.269453 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91410453-bea3-474c-88ee-4dbe04cdd94f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:50.286766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:50.303667 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:50.304390 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.304675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.304954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.317118 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:57:50.325518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:50.326681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:50.327604 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 635/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:50.332877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.410672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:50.428887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af83191-37c5-4e62-a7a6-c11493abc4b0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:50.462496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:50.463184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.463467 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.463740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.464475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:50.464783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:50.466172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:50.474096 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 642/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:50.479835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.581271 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cee9c-ddae-4341-9139-533418d822b3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:50.614675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:50.615470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.615759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.616039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.627194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:50.628645 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:50.645451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:50.646574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:50.647517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:50.647517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:50.647517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:50.647517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:50.647517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 636/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:50.652523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.750867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82d619-9561-499f-837a-0d5ab1909652 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:50.782640 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:50.783300 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.783588 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.783871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.784596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:50.784908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:50.786302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:50.798266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 643/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:50.804061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.836611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:50.915428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c053b5-4889-4ec6-8081-4fa87cc2cdab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:50.948910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:50.949733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:50.950024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:50.950305 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:50.957593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:50.959315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:57:50.976762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:50.978125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:50.979303 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 637/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:50.986596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:50.990389 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:51.084610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b69a4-b7d3-4c7c-bf01-5024fc3e94ef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:51.119669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:51.120276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:51.120565 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:51.120847 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:51.121713 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:51.122034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:51.123482 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:51.140898 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 644/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 11 11:57:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:51.146578 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:51.239967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:51.249608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c55c6-95c0-4958-8d92-7b0a50ab957b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:51.283554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:51.284206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:51.284495 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:51.284775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:51.351658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:51.353115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:51.438104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:51.449086 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 638/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 11 11:57:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:57:51.454834 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:51.518003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:51.556726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28141b4a-5fa2-4e05-8973-86671cf5f389 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:51.590853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:51.591650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:51.591936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:51.592215 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:51.602387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:51.603874 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:57:51.621175 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:51.622136 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:51.623148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:57:51.623148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:57:51.623148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:57:51.623148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:51.623148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 645/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 11:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:51.628133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:51.725113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255919-2d45-4c3a-bda8-58f4f049f3a3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:51.757373 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:51.758055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:51.758339 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:51.758614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:51.759333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:51.759638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:51.761546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:51.775197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 639/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 11 11:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:51.780941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:51.880806 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0f561-a94e-4ef1-aff0-5ed165d5183f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:51.895197 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:51.914973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:51.915774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:51.916066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:51.916348 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:51.922940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:51.924384 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:57:51.938432 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:51.939478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jul 11 11:57:51.940539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 646/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 11 11:57:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:51.945467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:51.995456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:52.047797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae584e0-4635-44ea-82a1-56e28391808c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:52.080971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:52.081625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:52.081939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:52.082217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:52.082996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:52.083555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:52.084974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9817-ef92-47fa-a194-1a9613abf0be None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:52.096913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 640/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 11 11:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:52.103188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:52.205049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5410b1-5c33-43c4-aa7a-be5245c8e466 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:52.238664 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:52.239316 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:52.239605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:52.239889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:52.240719 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] RBAC: Adding query filter params (role.id=40fbb4c110e445ea922bbfb223d8701c, scope.project.id=c2af7f192c8d41dfac1733b0da80cd73, user.id=a6db07035b4d472ebc58509502b666ba) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:52.241035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:52.242489 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d779ed1-f468-4a8a-987f-937af09c4706 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:52.249286 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 647/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 11 11:57:52 2022] GET /identity/v3/role_assignments?user.id=a6db07035b4d472ebc58509502b666ba&scope.project.id=c2af7f192c8d41dfac1733b0da80cd73&role.id=40fbb4c110e445ea922bbfb223d8701c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:54.291595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf675c89-14ad-4b59-8aaa-3b7ee236ff96 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:54.292164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf675c89-14ad-4b59-8aaa-3b7ee236ff96 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:54.292698 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf675c89-14ad-4b59-8aaa-3b7ee236ff96 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:54.294413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 641/1288] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:54.299647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:54.300069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:54.300201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:54.317360 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:54.318563 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:54.348366 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:54.385836 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 648/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:57:54.408353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf675c89-14ad-4b59-8aaa-3b7ee236ff96 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:54.508586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf675c89-14ad-4b59-8aaa-3b7ee236ff96 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:54.519584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:54.543394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:54.544194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:54.544482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:54.544761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:54.545410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:54.546808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:54.558655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:54.559834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:54.560907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 642/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:57:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:54.565860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:54.661647 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064cc72-833a-42c2-beeb-a7455588f8b2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:54.694930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:54.695581 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:54.695863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:54.696141 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:54.696840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:54.697772 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:54.699132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538589ba-8b92-4b48-9e9d-212bca2ead06 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:54.703901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 649/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 11 11:57:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:54.709942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:54.809124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1941-e27a-4d60-a23e-30e94fb69391 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:54.848513 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:54.849177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:54.849461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:54.849962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:54.850875 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:54.851268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:54.852864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49018c-3fe6-4a52-9390-2f0bf250eef1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:54.858727 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 643/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 11 11:57:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:56.892871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22203b2-4047-49c1-9c01-dcbeac74ce1e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:56.893475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22203b2-4047-49c1-9c01-dcbeac74ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:56.893475 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22203b2-4047-49c1-9c01-dcbeac74ce1e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:56.895036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 650/1293] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:56.900984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deb9a-18f6-4133-81f0-2aa2c05202b9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:56.901711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deb9a-18f6-4133-81f0-2aa2c05202b9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:56.901711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deb9a-18f6-4133-81f0-2aa2c05202b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:56.919430 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5f7deb9a-18f6-4133-81f0-2aa2c05202b9 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:56.920654 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5f7deb9a-18f6-4133-81f0-2aa2c05202b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:56.956206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:57.021104 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 644/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:57:57.044740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22203b2-4047-49c1-9c01-dcbeac74ce1e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:57.076140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:57.148953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22203b2-4047-49c1-9c01-dcbeac74ce1e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:57.182767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:57.183424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:57.183706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:57.183983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:57.184712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:57.186110 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf55424-13f2-4a0a-80ab-35f49400de27 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:57.200692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 651/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 11 11:57:57 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:57:59.295281 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c30aef-2ecd-487e-8d64-2c5c57822049 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:59.295842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c30aef-2ecd-487e-8d64-2c5c57822049 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:59.295842 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c30aef-2ecd-487e-8d64-2c5c57822049 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:59.297207 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 645/1296] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:57:59.302891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:59.303417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:59.303569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:59.321593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:59.323425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:59.400244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 652/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:57:59 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:57:59.423500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c30aef-2ecd-487e-8d64-2c5c57822049 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:59.528736 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c30aef-2ecd-487e-8d64-2c5c57822049 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:59.561959 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:59.562777 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:59.563058 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:59.563332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:59.563984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:59.565347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:59.578332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:59.579731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:57:59.580731 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 646/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:57:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:57:59.586782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:59.605750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:59.691220 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc4822-be93-43ed-bd9f-13658518be1b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:59.729578 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:59.730280 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:59.730568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:59.730849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:59.731575 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:59.731886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:59.733302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:59.739342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 653/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 11 11:57:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:59.746397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:59.780300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:59.878249 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b887e-bd52-4ce6-acdb-a9938f5a39db None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:59.923901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:59.924901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:59.925313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:59.925768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:59.927102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] RBAC: Adding query filter params (interface=public, service_id=764f4b5066744e47a08fc87a377f8352, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:59.927587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:59.929835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584abb9d-a011-4184-b164-56adae44a29f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:59.937097 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 647/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 11 11:57:59 2022] GET /identity/v3/endpoints?service_id=764f4b5066744e47a08fc87a377f8352&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:57:59.943292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:58:00.036148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:58:00.055455 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375ab9e-d227-41d0-a859-2ee6c7d59dba None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:58:00.090353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:58:00.091053 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:00.091339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:00.091616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:00.092335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:58:00.093782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5b5c6-42ca-4b50-9819-339006e2357c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:58:00.101648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 654/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 11 11:57:59 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:58:02.389773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624929b8-a0cf-4ada-b862-06fa4e9fac2c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:02.390393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624929b8-a0cf-4ada-b862-06fa4e9fac2c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:02.390393 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624929b8-a0cf-4ada-b862-06fa4e9fac2c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:02.391624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 648/1302] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:58:02.397226 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:02.397850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:02.397850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:02.416983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:58:02.418223 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:58:02.493473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 655/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:58:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:58:02.516819 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624929b8-a0cf-4ada-b862-06fa4e9fac2c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:58:02.647983 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:58:02.658593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624929b8-a0cf-4ada-b862-06fa4e9fac2c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:58:02.714350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:58:02.715629 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:02.716124 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:02.716569 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:02.717428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:58:02.719483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:58:02.739665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:58:02.741142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jul 11 11:58:02.742063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 649/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 11 11:58:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:58:02.748262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:58:02.827081 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:58:02.851808 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae34dc3-61f3-43f4-83fd-3a8662e5c52a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:58:02.890668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:58:02.891474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:02.891856 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:02.892272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:02.893267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:58:02.893694 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:58:02.895230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5e414-3c9f-433d-9a90-6a0aac7860a7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:58:02.900993 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 656/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 11 11:58:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:58:02.907198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:58:03.016030 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e18922-e4be-4704-8a2c-ac5eda942ed6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:58:03.051689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:58:03.052351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:58:03.052635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:58:03.052954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:58:03.053852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:58:03.055295 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c931f8-1141-4417-ac1b-91c45845a4ec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:58:03.074308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 650/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 11 11:58:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:58:12.897461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=30 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:58:13.069335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:09.399990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f943-dca0-4592-8e39-d9796b103a74 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.400970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f943-dca0-4592-8e39-d9796b103a74 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.401420 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f943-dca0-4592-8e39-d9796b103a74 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.408687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900cb2-c0a9-4383-97a6-09541653e5af None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.409574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900cb2-c0a9-4383-97a6-09541653e5af None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.410037 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900cb2-c0a9-4383-97a6-09541653e5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.412427 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 657/1307] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:09.502975 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cd900cb2-c0a9-4383-97a6-09541653e5af None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:09.504476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cd900cb2-c0a9-4383-97a6-09541653e5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.698408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 651/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:09.735046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.735335 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.735632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.737600 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 658/1309] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 11 11:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:09.745846 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc36ab-7d7c-4f4c-ac24-2f071ef33215 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.746644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc36ab-7d7c-4f4c-ac24-2f071ef33215 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.747064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc36ab-7d7c-4f4c-ac24-2f071ef33215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.915757 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-89cc36ab-7d7c-4f4c-ac24-2f071ef33215 None None] MFA Rules not processed for user `b5a31865c8ed4ab78bfdb17780b61d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:09.917455 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-89cc36ab-7d7c-4f4c-ac24-2f071ef33215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.956115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:10.036163 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.038063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.047588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 652/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 11 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:10.156063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a090b-89ca-4887-a4fd-4193b6ebaebf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.243516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.244186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.244474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.244753 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.246439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.308072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:10.370596 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.372258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346f949-3b3d-454e-bdb0-6316271d02ac service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.393010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 659/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:12.761635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.762171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.762285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.763609 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 653/1312] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:12.768791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f03f7-12c2-4a6d-891f-efd32cf919f6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.769184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f03f7-12c2-4a6d-891f-efd32cf919f6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.769351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f03f7-12c2-4a6d-891f-efd32cf919f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.843438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-677f03f7-12c2-4a6d-891f-efd32cf919f6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:12.844662 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-677f03f7-12c2-4a6d-891f-efd32cf919f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:12.947750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 660/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:12.987292 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.988833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.093333 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b1856-adf0-476b-8389-93e6aa993b89 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:13.166075 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:13.168795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:13.169718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.170186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.170687 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.172491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.287282 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:13.288801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b3ed7-31bd-47dd-a8c8-3789338a392a service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:13.307502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 654/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:15.729927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c305-d146-472e-b64c-1b6f0896dff5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.730531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c305-d146-472e-b64c-1b6f0896dff5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.730531 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c305-d146-472e-b64c-1b6f0896dff5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.731899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 661/1315] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:15.737031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1079-5821-48a6-9937-6ffff0aab762 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.737447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1079-5821-48a6-9937-6ffff0aab762 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.737611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1079-5821-48a6-9937-6ffff0aab762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.803071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cbae1079-5821-48a6-9937-6ffff0aab762 None None] MFA Rules not processed for user `d41522a9f214417fad9710b6bfce9f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:15.804847 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cbae1079-5821-48a6-9937-6ffff0aab762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:15.857889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:15.907606 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 655/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:15.963066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.963468 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.963759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.965154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 662/1317] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 11 11:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:15.972809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34448-7ad5-4372-a613-3f1b294ca0dc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.973313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34448-7ad5-4372-a613-3f1b294ca0dc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.973888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34448-7ad5-4372-a613-3f1b294ca0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.070762 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f4c34448-7ad5-4372-a613-3f1b294ca0dc None None] MFA Rules not processed for user `b5a31865c8ed4ab78bfdb17780b61d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:16.072023 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f4c34448-7ad5-4372-a613-3f1b294ca0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.155812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 656/1318] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:16.165972 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.167548 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.178910 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:16.278521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2efd-d2a4-421b-b404-500a70eebbac None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.375213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.375949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.376239 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.376516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.378222 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.443410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:16.504599 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.506130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff12cc1-c804-4b9e-8a9b-18baf0ce4f6a service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.527249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 663/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:19.536191 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:19.536775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:19.536775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:19.538789 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 657/1320] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:19.544207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ffc3-9520-44ea-b196-aeccc504286b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:19.544682 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ffc3-9520-44ea-b196-aeccc504286b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:19.544798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ffc3-9520-44ea-b196-aeccc504286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:19.612283 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5454ffc3-9520-44ea-b196-aeccc504286b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:19.613500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5454ffc3-9520-44ea-b196-aeccc504286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:19.714920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 664/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:19.753278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:19.754840 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:19.805599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:19.863639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0b08-d38d-49f8-aff4-49c928fa6291 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:19.937560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:19.938258 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:19.938547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:19.938820 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:19.940451 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:20.035111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:20.048812 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:20.050351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db3e92-fe80-46a8-9647-5efbc5005621 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:20.068343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 658/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:22.898576 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.899143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.899143 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.900539 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 665/1323] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:22.910587 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd458-620c-48a5-9d39-a38a78779f01 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.911107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd458-620c-48a5-9d39-a38a78779f01 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.911284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edd458-620c-48a5-9d39-a38a78779f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.977028 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b6edd458-620c-48a5-9d39-a38a78779f01 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:22.978268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b6edd458-620c-48a5-9d39-a38a78779f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.078721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 659/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:23.114508 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:23.116128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.139156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:23.247877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d470e-5885-4810-bfe1-d9daf4b5e195 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:23.335798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:23.336451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:23.336732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:23.337002 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:23.338699 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.423017 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:23.469368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:23.470878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5b48f-0651-4fd2-a6d4-dcf51aeac7fe service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:23.491303 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 666/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:25.746575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.747136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.747136 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.748513 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 660/1326] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:25.753820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caec8b-b268-497e-b45b-902377839406 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.754256 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caec8b-b268-497e-b45b-902377839406 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.754401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caec8b-b268-497e-b45b-902377839406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.822387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d1caec8b-b268-497e-b45b-902377839406 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.823569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d1caec8b-b268-497e-b45b-902377839406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.924262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 667/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.960212 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.961799 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.025406 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.072699 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc6da6-330c-49ec-92c6-2f12c80d2ae1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.148568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.149227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.149516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.149838 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.151516 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.266349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.268957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.270872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efca797-40d2-4fd7-8ed3-c2a236748930 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.290831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 661/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:28.789927 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.790500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.790500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.791876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 668/1329] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:28.797324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df68db-1eaa-4caa-80c5-0d97f577079a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.797835 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df68db-1eaa-4caa-80c5-0d97f577079a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.797932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df68db-1eaa-4caa-80c5-0d97f577079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.872658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b1df68db-1eaa-4caa-80c5-0d97f577079a None None] MFA Rules not processed for user `d41522a9f214417fad9710b6bfce9f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:28.873894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b1df68db-1eaa-4caa-80c5-0d97f577079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.975544 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 662/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:29.013334 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:29.014992 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:29.050878 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:29.127900 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50ac8-258b-4913-9c11-1e7f34412238 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:29.203749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:29.204322 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:29.204615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:29.204899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:29.206579 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:29.289326 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:29.323337 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:29.324799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9b68-d699-4e49-bf93-1825d50d30fc service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:29.343441 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 669/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:31.649365 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.650046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.650046 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.651343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 663/1332] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:31.657478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc653-439b-4926-91ca-68cad0ff6743 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.658094 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc653-439b-4926-91ca-68cad0ff6743 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.658213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dc653-439b-4926-91ca-68cad0ff6743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.733605 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7f2dc653-439b-4926-91ca-68cad0ff6743 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:31.734866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7f2dc653-439b-4926-91ca-68cad0ff6743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.840240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 670/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:31.877636 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:31.879335 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.950750 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:31.986359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546fe47-e9d6-4e9e-a070-76c89abf0e9e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.061703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.062334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.062618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.062894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.064539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.175697 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.177165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf955e4-41d0-4a66-af44-c91045642268 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.188740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:32.204557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 664/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:34.321779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.322351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.322351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.323817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 671/1335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:34.329297 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395f8a-e23c-489d-903a-14a60ed7b610 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.329817 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395f8a-e23c-489d-903a-14a60ed7b610 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.329915 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395f8a-e23c-489d-903a-14a60ed7b610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.402017 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-04395f8a-e23c-489d-903a-14a60ed7b610 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:34.403180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-04395f8a-e23c-489d-903a-14a60ed7b610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.544948 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 665/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:34.585636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:34.587201 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.628863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:34.696177 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead12a7-6430-4768-b981-9047a6762dde None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:34.770948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:34.771636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.771922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.772200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.773922 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.864566 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:34.893511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:34.895003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8253a6-bd2d-4b0a-9114-6665cf4de27e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:34.914552 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 672/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:37.357590 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:37.358236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:37.358236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:37.359645 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 666/1338] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:37.365321 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778681-0163-48a6-8bc0-1b417af2e389 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:37.365820 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778681-0163-48a6-8bc0-1b417af2e389 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:37.365936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778681-0163-48a6-8bc0-1b417af2e389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:37.433630 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eb778681-0163-48a6-8bc0-1b417af2e389 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:37.434825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb778681-0163-48a6-8bc0-1b417af2e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:37.534340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 673/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:37.570743 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:37.572366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:37.648683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:37.676952 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52d0c0-a057-42d5-96d9-bb4ea0cedc39 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:37.757156 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:37.757863 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:37.758152 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:37.758431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:37.760230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:37.893119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:37.894625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a44d9-6818-4853-8715-fb68331ba8f4 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:37.914473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 667/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:42.131456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.131990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.131990 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.137632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 674/1341] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:42.139230 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e502d-7e3c-4ec2-9512-a58ed9868685 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.139652 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e502d-7e3c-4ec2-9512-a58ed9868685 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.139942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e502d-7e3c-4ec2-9512-a58ed9868685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.144222 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:42.214800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ad3e502d-7e3c-4ec2-9512-a58ed9868685 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:42.215907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ad3e502d-7e3c-4ec2-9512-a58ed9868685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:42.319662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 668/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:42.360610 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:42.362192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:42.417185 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:42.482036 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8f48-8ccb-4cb3-99b4-0f54e463d825 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:42.563732 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:42.564394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.564684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.564964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.566659 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:42.672716 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:42.692049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:42.693569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba9122-b776-4989-9aca-38b98e017b04 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:42.713987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 675/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:45.606723 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:45.607291 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:45.607291 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:45.608972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 669/1344] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:45.615615 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f2eb-9ca6-4ccc-8aba-2169cae6b71d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:45.616172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f2eb-9ca6-4ccc-8aba-2169cae6b71d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:45.616172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f2eb-9ca6-4ccc-8aba-2169cae6b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:45.692876 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-63c8f2eb-9ca6-4ccc-8aba-2169cae6b71d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:45.694306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-63c8f2eb-9ca6-4ccc-8aba-2169cae6b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:45.803232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 676/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:45.844988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:45.848316 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:45.950876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:45.973429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5a624-1f6f-41d0-bc45-11d822dcf59c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:46.059655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:46.060348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:46.060625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:46.060910 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:46.062641 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:46.197018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:46.199002 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54625a-b4f9-40d4-ad00-ca4e2c1e1f53 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:46.218725 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 670/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:51.377870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:51.378376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:51.378757 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:51.380299 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 677/1347] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 11:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:51.386372 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a966-e5d7-4831-b39a-8129dfa3c826 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:51.387201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a966-e5d7-4831-b39a-8129dfa3c826 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:51.387366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a966-e5d7-4831-b39a-8129dfa3c826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:51.403370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:51.468025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8760a966-e5d7-4831-b39a-8129dfa3c826 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:51.469284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8760a966-e5d7-4831-b39a-8129dfa3c826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:51.580770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 671/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:51.625265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:51.626880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:51.691759 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:51.746825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5df99a-5702-4863-8be7-b4266e0c5ea6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:51.834106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:51.834779 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:51.835076 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:51.835355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:51.837071 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:51.947517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:51.951788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:51.953392 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a1689-debc-4a11-b535-b070a2f44af8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:51.973162 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 678/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:57.168781 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01f79-8f6d-4c54-8838-c5a80ccb598b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:57.171794 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01f79-8f6d-4c54-8838-c5a80ccb598b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:57.172574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01f79-8f6d-4c54-8838-c5a80ccb598b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:57.174324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 672/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 11 11:59:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:59:57.183156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7960736-e796-4501-bec8-f2f084f372ac None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:57.183156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7960736-e796-4501-bec8-f2f084f372ac None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:57.183156 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7960736-e796-4501-bec8-f2f084f372ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:57.287038 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b7960736-e796-4501-bec8-f2f084f372ac None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:57.288171 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b7960736-e796-4501-bec8-f2f084f372ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:57.495295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 679/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 11 11:59:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:01.578994 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=59 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:02.269566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7100-b144-471b-9caf-9ae72fb62d25 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:02.269921 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7100-b144-471b-9caf-9ae72fb62d25 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:02.270195 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f7100-b144-471b-9caf-9ae72fb62d25 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:02.271675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 673/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 11 12:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:02.279189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d7e9-74c9-40f3-a30a-e2de3f1c6691 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:02.279704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d7e9-74c9-40f3-a30a-e2de3f1c6691 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:02.279812 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d7e9-74c9-40f3-a30a-e2de3f1c6691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:02.345636 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:02.375387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ec44d7e9-74c9-40f3-a30a-e2de3f1c6691 None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:02.376801 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ec44d7e9-74c9-40f3-a30a-e2de3f1c6691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:02.515648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 680/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 11 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:04.085038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a5c6-02b2-4a57-9011-ee6ec59deb37 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:04.085849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a5c6-02b2-4a57-9011-ee6ec59deb37 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:04.085849 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a5c6-02b2-4a57-9011-ee6ec59deb37 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:04.087016 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 674/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 11 12:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:04.093765 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92452013-57d1-4b31-ac0c-638e3a4009a1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:04.094532 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92452013-57d1-4b31-ac0c-638e3a4009a1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:04.094838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92452013-57d1-4b31-ac0c-638e3a4009a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:04.217239 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-92452013-57d1-4b31-ac0c-638e3a4009a1 None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:04.219282 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92452013-57d1-4b31-ac0c-638e3a4009a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:04.294995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:04.370207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 681/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 11 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:08.280465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303952a-63c8-47a9-82e6-04227d95ad2a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:08.281211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303952a-63c8-47a9-82e6-04227d95ad2a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:08.281211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303952a-63c8-47a9-82e6-04227d95ad2a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:08.282664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 675/1356] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:08.298050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d8c1-92ec-4587-b4fc-c2858607644e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:08.298050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d8c1-92ec-4587-b4fc-c2858607644e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:08.298888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d8c1-92ec-4587-b4fc-c2858607644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:08.402500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6046d8c1-92ec-4587-b4fc-c2858607644e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:08.403945 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6046d8c1-92ec-4587-b4fc-c2858607644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:08.573973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 682/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:08.639149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:08.640671 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:08.641500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:08.644919 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 676/1358] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 11 12:00:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:08.664497 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e80f-f995-4eb5-913a-d8453b76a104 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:08.665839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e80f-f995-4eb5-913a-d8453b76a104 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:08.666632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e80f-f995-4eb5-913a-d8453b76a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:08.696683 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:08.820227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aa19e80f-f995-4eb5-913a-d8453b76a104 None None] MFA Rules not processed for user `80890a6017284bf090892fbcbb618f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:08.822628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa19e80f-f995-4eb5-913a-d8453b76a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:09.046037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 683/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 11 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:09.059984 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:09.061807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:09.246772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb34221-abb2-4dd9-ae45-a83f1b80e82d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:09.377211 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:09.452401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:09.453518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:09.454127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:09.454880 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:09.457398 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:09.596159 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:09.599662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d719df9-85ea-40b6-9a93-63224e6433c4 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:09.633741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 677/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:12.362320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5875a-4988-435b-840f-821a30599213 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.362320 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5875a-4988-435b-840f-821a30599213 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.363568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5875a-4988-435b-840f-821a30599213 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.369227 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 684/1361] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:12.370299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0d87c-3866-457e-87cc-d6a0d85a5053 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.370299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0d87c-3866-457e-87cc-d6a0d85a5053 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.371061 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0d87c-3866-457e-87cc-d6a0d85a5053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.445665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4cf0d87c-3866-457e-87cc-d6a0d85a5053 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:12.446887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4cf0d87c-3866-457e-87cc-d6a0d85a5053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:12.518100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:12.609389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 678/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:12.666260 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5875a-4988-435b-840f-821a30599213 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:12.668139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-57d5875a-4988-435b-840f-821a30599213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:12.763285 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:12.901977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5875a-4988-435b-840f-821a30599213 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:13.080924 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:13.081609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:13.081953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:13.082184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:13.085343 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:13.122979 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:13.234977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:13.236593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906527ff-cff8-45eb-a49d-853dc523016a service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:13.262236 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 685/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:15.742823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3ac8f-b759-42ae-bebe-b7043906e1cf None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:15.743378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3ac8f-b759-42ae-bebe-b7043906e1cf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:15.743378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3ac8f-b759-42ae-bebe-b7043906e1cf None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:15.745448 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 679/1364] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:15.756867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70688-bedf-471e-a75d-be2abad5e221 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:15.757888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70688-bedf-471e-a75d-be2abad5e221 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:15.757888 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c70688-bedf-471e-a75d-be2abad5e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:15.843196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-20c70688-bedf-471e-a75d-be2abad5e221 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:15.844637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-20c70688-bedf-471e-a75d-be2abad5e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:15.955597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:15.961735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 686/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:16.036427 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:16.036867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:16.037440 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:16.039473 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 680/1366] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 11 12:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:16.048724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a31f-69ed-4838-a01e-46cffa2e7fbf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:16.049270 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a31f-69ed-4838-a01e-46cffa2e7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:16.049747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a31f-69ed-4838-a01e-46cffa2e7fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:16.163580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d991a31f-69ed-4838-a01e-46cffa2e7fbf None None] MFA Rules not processed for user `80890a6017284bf090892fbcbb618f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:16.165085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d991a31f-69ed-4838-a01e-46cffa2e7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:16.379388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 687/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 11 12:00:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:16.394474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:16.397066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:16.634420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90edcb9-e194-4d54-93ff-f62ac1f852eb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:16.673608 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:16.854188 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:16.855384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:16.855887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:16.856347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:16.859628 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:17.021150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:17.023100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138ef2e-3511-4f8b-afb9-b12f1d63c093 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:17.044848 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 681/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:17.307249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d1e18-9f05-4796-ab69-c3418e344a07 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:17.308021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d1e18-9f05-4796-ab69-c3418e344a07 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:17.308021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d1e18-9f05-4796-ab69-c3418e344a07 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:17.309488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 688/1369] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:00:17.316385 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238b3a7-089d-4a24-9e4a-74ff537addbc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:17.317942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238b3a7-089d-4a24-9e4a-74ff537addbc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:17.318554 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238b3a7-089d-4a24-9e4a-74ff537addbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:17.378793 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:17.446331 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5238b3a7-089d-4a24-9e4a-74ff537addbc None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:17.447888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5238b3a7-089d-4a24-9e4a-74ff537addbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:17.599633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 682/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:17.959872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:17.959872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:17.959872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:17.959872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 689/1371] 10.222.0.57 () {54 vars in 963 bytes} [Mon Jul 11 12:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:17.965200 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585654a-cd65-4ced-b113-a5b600134f06 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:17.965867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585654a-cd65-4ced-b113-a5b600134f06 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:17.965867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585654a-cd65-4ced-b113-a5b600134f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.054588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4585654a-cd65-4ced-b113-a5b600134f06 None None] MFA Rules not processed for user `0cf7137aab9d4ebb8b83677cc4b0d817`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:18.055980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4585654a-cd65-4ced-b113-a5b600134f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.127068 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.163140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 683/1372] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Jul 11 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:18.171288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:18.173010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.253561 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.334122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3ec86-9e48-4da8-b18d-b74f70e85dc5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:18.415923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:18.416583 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.416865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.417148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.418980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.527145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.542839 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:18.544439 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09da4af-9f86-479a-86fe-c86b3c51886c service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:18.567054 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 690/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:19.619337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.620051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.620051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.621278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 684/1374] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:19.627485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da671-b944-4f20-8376-a5b805a7ca5c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.628003 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da671-b944-4f20-8376-a5b805a7ca5c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.628157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da671-b944-4f20-8376-a5b805a7ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.701589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f51da671-b944-4f20-8376-a5b805a7ca5c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:19.702902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f51da671-b944-4f20-8376-a5b805a7ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.826235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 691/1375] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:19.862893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.864680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.054642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a58ae-34b0-4448-b04c-38ba40885939 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.133630 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:20.218929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.219574 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.219913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.220276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.222244 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.354261 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.356558 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d1c0f-f36f-4e20-b481-18e5a5cc4a50 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:20.378470 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 685/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:22.536763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.537336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.537336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.539233 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 692/1377] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:22.544991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc74-c6e8-4aa3-b59c-8db98b2761dd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.545481 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc74-c6e8-4aa3-b59c-8db98b2761dd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.545580 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc74-c6e8-4aa3-b59c-8db98b2761dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.631631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b6cddc74-c6e8-4aa3-b59c-8db98b2761dd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:22.632894 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b6cddc74-c6e8-4aa3-b59c-8db98b2761dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:22.640757 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:22.742708 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 686/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:22.790626 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.792340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:22.897916 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:23.012085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9ded-563b-458b-ad15-73dbde313ead None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:23.184677 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:23.185618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:23.186152 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:23.186535 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:23.188523 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:23.211923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:23.315652 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:23.318376 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b9d0-5972-48b8-9de9-08ea1c3a6f18 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:23.343009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 693/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:25.523557 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:25.524149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:25.524805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:25.526907 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 687/1380] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:25.532688 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e38dee-dc58-4b0a-92a7-ccb4b532d956 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:25.533308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e38dee-dc58-4b0a-92a7-ccb4b532d956 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:25.534359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e38dee-dc58-4b0a-92a7-ccb4b532d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:25.622121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-92e38dee-dc58-4b0a-92a7-ccb4b532d956 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:25.623387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92e38dee-dc58-4b0a-92a7-ccb4b532d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:25.738788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:25.751968 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 694/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:25.796891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:25.798946 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:25.999622 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:26.012742 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4152-2fcc-4181-9ba7-7dbe048a3b4a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.165092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.165951 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.166344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.166871 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.168746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:26.293582 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.295206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b43f-ff37-44a7-96da-e2a92505ebfd service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.310144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:26.323280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 688/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:28.478060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:28.478678 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:28.479007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:28.481941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 695/1383] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:28.491221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386960a-f42d-47b8-b970-b050353e859f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:28.491831 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386960a-f42d-47b8-b970-b050353e859f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:28.491944 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386960a-f42d-47b8-b970-b050353e859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:28.586096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6386960a-f42d-47b8-b970-b050353e859f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:28.587549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6386960a-f42d-47b8-b970-b050353e859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:28.695020 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 689/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:28.738962 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:28.741308 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:28.932756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7507a7-629d-4892-abe7-1207079bc50c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:29.017237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:29.080845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:29.082075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:29.082685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:29.083480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:29.086705 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:29.215954 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:29.219183 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72403df-c46a-4c19-b747-b64589c10efc service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:29.255030 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 696/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:31.386142 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.387057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.387320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.390094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 690/1386] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:31.395791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd178-9103-4224-89ad-ba10ce066860 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.396629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd178-9103-4224-89ad-ba10ce066860 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.396629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd178-9103-4224-89ad-ba10ce066860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.474852 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-66efd178-9103-4224-89ad-ba10ce066860 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:31.476133 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-66efd178-9103-4224-89ad-ba10ce066860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:31.541724 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:31.629062 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 697/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:31.678539 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.682768 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:31.827064 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:31.916500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a58c0-0019-4d9b-b95b-f84442110f55 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.066901 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.068532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.068532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.068823 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.070394 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:32.122071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:32.187141 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:32.188770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0efe57-8bdf-4190-aa82-012d80928ae8 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.212052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 691/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:34.495407 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:34.495948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:34.495948 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:34.497375 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 698/1389] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:34.503115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17f56-446a-44da-82d0-d472300b6a80 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:34.503617 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17f56-446a-44da-82d0-d472300b6a80 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:34.503712 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a17f56-446a-44da-82d0-d472300b6a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:34.586805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-65a17f56-446a-44da-82d0-d472300b6a80 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:34.588047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-65a17f56-446a-44da-82d0-d472300b6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:34.698351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 692/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:34.746105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:34.747620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:34.927975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15326-f0f1-466e-b88a-a63a52b8e946 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:34.952052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:35.066849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:35.067955 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:35.068245 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:35.068526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:35.070242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:35.187912 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:35.189570 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5abb0f-81ed-47d7-8901-ca56f5bf118c service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:35.211864 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 699/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:36.269293 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9241-96c2-43bd-8482-2ec4268965e5 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.269997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9241-96c2-43bd-8482-2ec4268965e5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.269997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9241-96c2-43bd-8482-2ec4268965e5 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.271784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 693/1392] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:36.277729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc828-988c-4c24-86bf-809734e2c65f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.278313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc828-988c-4c24-86bf-809734e2c65f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.278313 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc828-988c-4c24-86bf-809734e2c65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.312413 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:36.359838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b5ffc828-988c-4c24-86bf-809734e2c65f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:36.361218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5ffc828-988c-4c24-86bf-809734e2c65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:36.465123 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 700/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:36.522981 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.523759 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.524285 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.527038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 694/1394] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 11 12:00:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:36.532546 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570b98-b624-4c00-baf1-5d9ee9f17dfa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.532840 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570b98-b624-4c00-baf1-5d9ee9f17dfa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.533121 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570b98-b624-4c00-baf1-5d9ee9f17dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.624148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-16570b98-b624-4c00-baf1-5d9ee9f17dfa None None] MFA Rules not processed for user `55a5560632ed452c8287a7c91b57e420`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:36.625518 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-16570b98-b624-4c00-baf1-5d9ee9f17dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:36.669934 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:36.710863 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 701/1395] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 11 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:36.718898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:36.720376 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:36.735358 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:36.832351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69327bc6-25cb-4fc5-9a56-11d459bc2e62 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:36.920102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55a5560632ed452c8287a7c91b57e420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:36.921284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.921856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.922389 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.925859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:37.018270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:37.076219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:37.077720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c659-3b0d-4c1c-a134-a70075acc13b service glance] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:37.097804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 695/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:37.432528 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:37.433084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:37.433084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:37.435802 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 702/1397] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:00:37.439841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056a65-2f29-4913-a4b3-9b98b2480fe3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:37.440306 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056a65-2f29-4913-a4b3-9b98b2480fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:37.440423 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056a65-2f29-4913-a4b3-9b98b2480fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:37.523269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f2056a65-2f29-4913-a4b3-9b98b2480fe3 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:37.523269 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f2056a65-2f29-4913-a4b3-9b98b2480fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:37.638642 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 696/1398] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:37.708685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:37.710678 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:37.866856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7139e-acfb-426b-88e3-337eafdc541f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:37.867201 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7139e-acfb-426b-88e3-337eafdc541f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:37.867405 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7139e-acfb-426b-88e3-337eafdc541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:37.885543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f40f63-6408-4c7a-9530-1336d729334b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:37.909954 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:37.953207 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:37.984826 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cfd7139e-acfb-426b-88e3-337eafdc541f None None] MFA Rules not processed for user `9c42d851cdef43fcb152473bebeb08f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:37.987069 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cfd7139e-acfb-426b-88e3-337eafdc541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.049266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:38.050031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:38.050292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:38.050619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:38.052853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.096112 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 697/1399] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 11 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:38.144644 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c497e2-fd83-4938-a221-880cd420407a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:38.144935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c497e2-fd83-4938-a221-880cd420407a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:38.145213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c497e2-fd83-4938-a221-880cd420407a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:38.146943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 698/1400] 10.222.0.57 () {54 vars in 942 bytes} [Mon Jul 11 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:00:38.156588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03ebe0-e858-4e5f-9eaf-74ebcd8ab165 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:38.156950 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03ebe0-e858-4e5f-9eaf-74ebcd8ab165 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:38.157158 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03ebe0-e858-4e5f-9eaf-74ebcd8ab165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:38.169790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:38.171431 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:38.198787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 703/1401] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:38.250119 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7b03ebe0-e858-4e5f-9eaf-74ebcd8ab165 None None] MFA Rules not processed for user `2ca0990c5479452da837df05b9fbcacb`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:38.251375 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7b03ebe0-e858-4e5f-9eaf-74ebcd8ab165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.335851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:38.408856 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 699/1402] 10.222.0.57 () {58 vars in 1039 bytes} [Mon Jul 11 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:38.421067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:38.423031 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.497574 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:38.535476 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f4f3b-9cf7-4e72-b6f8-eedcdeb5b32f service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:38.673607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ca0990c5479452da837df05b9fbcacb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:38.674804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:38.675336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:38.675865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:38.679853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.797548 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:38.799113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6c09e-19fa-411e-85cc-c8a314793612 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:38.805412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:38.810234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 704/1403] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 11 12:00:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 12:00:40.359165 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.360032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.360032 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.368453 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 700/1404] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:40.368763 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5600ca-3532-4f4e-87e9-c288656b372f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.369491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5600ca-3532-4f4e-87e9-c288656b372f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.369491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5600ca-3532-4f4e-87e9-c288656b372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.465969 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1d5600ca-3532-4f4e-87e9-c288656b372f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:40.467127 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1d5600ca-3532-4f4e-87e9-c288656b372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.617723 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 705/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:40.678741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.685551 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.903118 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383f2a8-103a-4b8e-8434-f8114f0fc7d4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.941210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:41.105540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.106223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.106701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.107194 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.109549 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.254870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.256976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d71e3-8018-4405-801a-c1585093a7dc service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.289929 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 701/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:43.503896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.504629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.504629 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.506453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 706/1407] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:00:43.512019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fa31-e3bd-4574-bb1e-2dca7053372d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.512497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fa31-e3bd-4574-bb1e-2dca7053372d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.512600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fa31-e3bd-4574-bb1e-2dca7053372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.560210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:43.599129 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3bc0fa31-e3bd-4574-bb1e-2dca7053372d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:43.601007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3bc0fa31-e3bd-4574-bb1e-2dca7053372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:43.741205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 702/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:43.817292 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:43.819442 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:43.956391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:44.063959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cda458-f38e-4a8a-926d-4dba2e0e6257 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:44.235478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:44.235478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:44.235478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:44.235478 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:44.236098 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:44.285359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:44.348204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:44.351033 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd790f51-39a8-4699-832c-2a54b765e48a service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:44.388504 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 707/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:53.738573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=51 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:54.383712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=27 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:01.304182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad7d4-599c-4116-9921-65ef519ffa07 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:01.304767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad7d4-599c-4116-9921-65ef519ffa07 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:01.304767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad7d4-599c-4116-9921-65ef519ffa07 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:01.306460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 703/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 11 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:01.315791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fbb31-a67e-4595-bcdc-0c92bb31377b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:01.315791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fbb31-a67e-4595-bcdc-0c92bb31377b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:01.315791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fbb31-a67e-4595-bcdc-0c92bb31377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:01.397902 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2d3fbb31-a67e-4595-bcdc-0c92bb31377b None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:01.399192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2d3fbb31-a67e-4595-bcdc-0c92bb31377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:01.518368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 708/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 11 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:01.533459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c512-b9d6-4fac-ab34-20c503f7b03a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:01.533782 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c512-b9d6-4fac-ab34-20c503f7b03a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:01.534065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372c512-b9d6-4fac-ab34-20c503f7b03a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:01.535435 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 704/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 11 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:01.541122 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ebc1-3105-45dd-b62f-45814e6d90cd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:01.541412 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ebc1-3105-45dd-b62f-45814e6d90cd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:01.541718 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ebc1-3105-45dd-b62f-45814e6d90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:01.620032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:01.623933 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5a39ebc1-3105-45dd-b62f-45814e6d90cd None None] MFA Rules not processed for user `80890a6017284bf090892fbcbb618f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:01.625951 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5a39ebc1-3105-45dd-b62f-45814e6d90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:01.750448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 709/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 11 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:04.260893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:04.261480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:04.261480 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:04.262935 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:04.268410 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56169c-a872-468e-baf4-eafdaa4fd0d8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:04.268849 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56169c-a872-468e-baf4-eafdaa4fd0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:04.268963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56169c-a872-468e-baf4-eafdaa4fd0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:04.337547 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ce56169c-a872-468e-baf4-eafdaa4fd0d8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:04.338774 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce56169c-a872-468e-baf4-eafdaa4fd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:04.391980 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:04.434684 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:04.478183 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:04.479775 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:04.629169 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129f91-497a-4f3c-962d-0aa031545b1c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:04.706065 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:04.747639 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:04.748296 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:04.748584 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:04.748864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:04.750576 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:04.863092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:04.865343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b038-0172-45d1-ac96-570ded2951bc service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:04.883870 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 706/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:05.795429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:05.796009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:05.796009 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:05.797336 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:05.802873 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf0ca-cc1b-48df-b923-adf6234ef897 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:05.803363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf0ca-cc1b-48df-b923-adf6234ef897 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:05.803460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf0ca-cc1b-48df-b923-adf6234ef897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:05.882411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-acacf0ca-cc1b-48df-b923-adf6234ef897 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:05.883614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-acacf0ca-cc1b-48df-b923-adf6234ef897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:05.910997 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:05.984275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:06.131589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:06.133140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:06.288005 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d74b6-6dce-483a-88c0-feb5a9277d09 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:06.329402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:06.405620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:06.406210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:06.406498 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:06.406780 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:06.408421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:06.499886 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:06.501480 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bbc47-819c-40c8-9d1f-7f8d6d22b929 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:06.520586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 712/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:07.404144 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:07.404720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:07.404720 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:07.406127 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:07.411817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67735-5c95-467c-90d3-46b8357da1d0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:07.412295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67735-5c95-467c-90d3-46b8357da1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:07.412404 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67735-5c95-467c-90d3-46b8357da1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:07.467244 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:07.490756 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-06e67735-5c95-467c-90d3-46b8357da1d0 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:07.491989 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-06e67735-5c95-467c-90d3-46b8357da1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:07.600880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:07.748171 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:07.749740 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:07.904669 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec43fd-9532-4b34-b337-23c99e62316c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:07.917449 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:08.023386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.023949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.024227 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.024497 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.026234 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:08.118902 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.120593 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40390dc-e4a5-4646-a062-b314ed121b1f service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.141503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 709/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:09.016176 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:09.016761 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:09.016890 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:09.018339 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:01:09.023992 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcf71-880f-4c75-b466-48040007ca0b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:09.024502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcf71-880f-4c75-b466-48040007ca0b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:09.024611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcf71-880f-4c75-b466-48040007ca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:09.043447 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:09.101280 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ae6bcf71-880f-4c75-b466-48040007ca0b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:09.102517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ae6bcf71-880f-4c75-b466-48040007ca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:09.198232 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:09.348520 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:09.350128 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:09.474189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:09.513157 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b319ea9-90c1-4b5d-8993-275d359df952 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:09.627586 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:09.628238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:09.628522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:09.628797 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:09.631065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:09.713184 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:09.727798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:09.729388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424627-ce57-4360-ae2c-7b20f6a9594e service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:09.748106 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 715/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:10.620947 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:10.621503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:10.621503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:10.622971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:10.628087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3847e-2cf7-453c-8a42-fe730e3dbdb6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:10.628573 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3847e-2cf7-453c-8a42-fe730e3dbdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:10.628672 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3847e-2cf7-453c-8a42-fe730e3dbdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:10.702620 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-92a3847e-2cf7-453c-8a42-fe730e3dbdb6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:10.703830 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92a3847e-2cf7-453c-8a42-fe730e3dbdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:10.801919 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:10.951917 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:10.953460 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:11.039872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:11.110276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeb735-c8fc-4b0f-bbce-c20a1a1e1436 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:11.232100 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:11.232670 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:11.232957 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:11.233233 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:11.234991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:11.291855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:11.333791 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:11.335677 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e450d8-49ea-4fe0-bf21-713a965d7f8e service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:11.355542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 712/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:12.181070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:12.181706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:12.181706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:12.183109 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:12.188351 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cee8c4-0521-4672-8dea-bd5a354aacb8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:12.188885 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cee8c4-0521-4672-8dea-bd5a354aacb8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:12.188885 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cee8c4-0521-4672-8dea-bd5a354aacb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:12.264003 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b4cee8c4-0521-4672-8dea-bd5a354aacb8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:12.265128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b4cee8c4-0521-4672-8dea-bd5a354aacb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:12.373019 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:01:12.525456 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:12.527032 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:12.577113 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:12.676425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a4e7-6150-4e68-9900-e5b5acc67ff1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:12.794507 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:12.795169 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:12.795462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:12.795747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:12.797484 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:12.806447 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:12.893221 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:12.895331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155d8a-be09-4ceb-a76c-f301a29dd421 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:12.913940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 718/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:13.723665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:13.724721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:13.724865 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:13.726437 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:13.731848 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde607e7-b23c-428e-b170-477388d6242d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:13.732329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde607e7-b23c-428e-b170-477388d6242d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:13.732449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde607e7-b23c-428e-b170-477388d6242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:13.805987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bde607e7-b23c-428e-b170-477388d6242d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:13.807212 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bde607e7-b23c-428e-b170-477388d6242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:13.876341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:13.907784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:14.056726 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:14.058276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:14.076436 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:14.212126 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df00b3e-291f-4642-a040-20cfabdf8496 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:14.302887 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:14.333969 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:14.334809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:14.335092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:14.335370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:14.337075 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:14.434734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:14.436365 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865fa3-2491-4146-9b2b-1a7dcb16c8ec service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:14.456197 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 715/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:15.264075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:15.264643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:15.264643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:15.266065 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:15.271572 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f041a-071d-4f6d-8932-a62d2554d11a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:15.272013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f041a-071d-4f6d-8932-a62d2554d11a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:15.272128 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f041a-071d-4f6d-8932-a62d2554d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:15.349733 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c30f041a-071d-4f6d-8932-a62d2554d11a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:15.350943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c30f041a-071d-4f6d-8932-a62d2554d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:15.392350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:15.447809 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:15.613414 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:15.614867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:15.759257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3b38-9f9f-4124-a79f-0753ae7c977a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:15.812828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:15.872158 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:15.872700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:15.872984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:15.873259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:15.874923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:15.964328 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:15.966037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8fc1-148b-4640-81c5-ee9a3023984b service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:15.984355 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 721/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:16.803092 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:16.803681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:16.803681 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:16.805057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:16.810754 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1afe-705c-4d06-a7b0-f2a4800a55aa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:16.811290 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1afe-705c-4d06-a7b0-f2a4800a55aa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:16.811409 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1afe-705c-4d06-a7b0-f2a4800a55aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:16.891418 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-038c1afe-705c-4d06-a7b0-f2a4800a55aa None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:16.893050 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-038c1afe-705c-4d06-a7b0-f2a4800a55aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:16.895788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:16.994687 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:17.145221 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:17.146867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:17.298112 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9ea2e-8a45-4aa5-88fb-71a341036522 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:17.325689 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:17.412503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:17.413062 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:17.413345 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:17.413619 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:17.415301 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:17.504090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:17.505773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304a031-03b9-48dd-adcd-3d9d8598fd55 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:17.524441 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 718/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:19.468590 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:19.469213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:19.469213 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:19.470602 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:19.476299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c36e4-e61c-4a7f-9ba0-f15256aa6f5d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:19.476784 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c36e4-e61c-4a7f-9ba0-f15256aa6f5d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:19.476883 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c36e4-e61c-4a7f-9ba0-f15256aa6f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:19.517925 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:19.543961 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ad4c36e4-e61c-4a7f-9ba0-f15256aa6f5d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:19.545180 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ad4c36e4-e61c-4a7f-9ba0-f15256aa6f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:19.650198 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:01:19.690525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:19.692069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:19.802191 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04540f4f-1e8a-4db3-a92e-374e6401d1b7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:19.835711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:19.876855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:19.877506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:19.877817 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:19.878099 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:19.879736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:19.991265 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:19.992721 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e38a3-ca53-4a8e-9324-72d681cbf614 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:20.011803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 724/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:22.274398 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:22.274967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:22.274967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:22.276346 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:22.287393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6d0e0-898b-42de-a1a6-f168ad974007 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:22.287928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6d0e0-898b-42de-a1a6-f168ad974007 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:22.287928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6d0e0-898b-42de-a1a6-f168ad974007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:22.356267 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6bb6d0e0-898b-42de-a1a6-f168ad974007 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:22.357494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6bb6d0e0-898b-42de-a1a6-f168ad974007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:22.370960 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:22.462307 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:22.499748 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:22.501474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:22.610599 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ad9ae-5be7-498d-bfa2-22fcbfd14c40 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:22.622924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:22.686785 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:22.687438 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:22.687729 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:22.688010 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:22.689728 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:22.802721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:22.804170 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b506878-5ba5-4631-a889-87d20240236c service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:22.824079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 721/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:25.191340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:25.191851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:25.191851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:25.193250 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:25.198977 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa1a09-658a-4e89-86cb-d69fb3350d67 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:25.199420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa1a09-658a-4e89-86cb-d69fb3350d67 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:25.199547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa1a09-658a-4e89-86cb-d69fb3350d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:25.248072 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:25.266533 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d4fa1a09-658a-4e89-86cb-d69fb3350d67 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:25.267752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d4fa1a09-658a-4e89-86cb-d69fb3350d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:25.365746 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:25.404178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:25.405845 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:25.519029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce635e-a5a5-4fa2-99bc-0937462cfa7e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:25.563453 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:25.603382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:25.604041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:25.604326 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:25.604603 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:25.606598 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:25.718354 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:25.719798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b00e2-1a81-4a47-b542-4e97d68dfb22 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:25.738391 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 727/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:27.921924 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:27.922476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:27.922476 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:27.923885 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:27.929295 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e7cf-3639-49d6-b452-1863fe56da0c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:27.929729 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e7cf-3639-49d6-b452-1863fe56da0c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:27.929906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e7cf-3639-49d6-b452-1863fe56da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:27.999866 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-00e2e7cf-3639-49d6-b452-1863fe56da0c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:28.001095 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-00e2e7cf-3639-49d6-b452-1863fe56da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:28.022778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:28.115506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:28.155277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:28.156843 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:28.272047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f74bdb-715f-45f6-bb11-8fc6c6b1fd37 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:28.292911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:28.348190 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:28.348872 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:28.349155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:28.349428 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:28.351103 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:28.460192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:28.461649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4d3ad-0c8a-4f8c-a3fb-d39c67d03a66 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:28.481106 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 724/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:30.588362 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:30.588936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:30.588936 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:30.590429 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:30.595899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07abe3-90f5-4824-8137-4ce1f8a174d1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:30.596414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07abe3-90f5-4824-8137-4ce1f8a174d1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:30.596529 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07abe3-90f5-4824-8137-4ce1f8a174d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:30.662942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:30.666208 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-af07abe3-90f5-4824-8137-4ce1f8a174d1 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:30.668051 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-af07abe3-90f5-4824-8137-4ce1f8a174d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:30.766763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:30.805827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:30.807425 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:30.924506 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7774a3-fcbb-4777-9f02-593fa2d78762 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:30.975329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:31.000011 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:31.000669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:31.000956 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:31.001235 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:31.002930 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:31.111768 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:31.113208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad7c3a-40c3-453a-a37f-c6ea9d570857 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:31.132342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 730/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:33.266668 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:33.267457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:33.267457 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:33.268949 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:33.274159 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa37e17-5878-4873-801b-0d4724fe3b48 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:33.274692 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa37e17-5878-4873-801b-0d4724fe3b48 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:33.274790 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa37e17-5878-4873-801b-0d4724fe3b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:33.341995 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9aa37e17-5878-4873-801b-0d4724fe3b48 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:33.343228 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9aa37e17-5878-4873-801b-0d4724fe3b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:33.373395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:33.448155 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:33.487683 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:33.489866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:33.597465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964785e5-9b7a-45c8-8bf8-193bcc12bdd9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:33.627487 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:33.673226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:33.673893 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:33.674179 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:33.674454 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:33.676094 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:33.798869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:33.800356 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e75b2-00f9-46ae-8968-9b12da904dc5 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:33.818678 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 727/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:35.932543 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:35.933097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:35.933097 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:35.934454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:35.939931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab843a-1b4a-453b-8373-4b29049bc159 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:35.940446 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab843a-1b4a-453b-8373-4b29049bc159 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:35.940562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab843a-1b4a-453b-8373-4b29049bc159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:36.008566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-11ab843a-1b4a-453b-8373-4b29049bc159 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:36.009761 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-11ab843a-1b4a-453b-8373-4b29049bc159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:36.018624 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:36.122553 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:36.160734 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:36.162987 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:36.266485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b119c8-b189-46f7-a084-8a7c7227e342 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:36.323438 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:36.343200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:36.343850 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:36.344140 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:36.344411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:36.346079 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:36.459262 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:36.460959 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e1531-3c1b-4110-aa3c-a90255e03422 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:36.479063 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 733/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:38.618653 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:38.619229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:38.619229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:38.620568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:38.626242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6c01-3e78-45f6-9733-e0e295130165 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:38.626804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6c01-3e78-45f6-9733-e0e295130165 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:38.626804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6c01-3e78-45f6-9733-e0e295130165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:38.699747 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-591a6c01-3e78-45f6-9733-e0e295130165 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:38.700981 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-591a6c01-3e78-45f6-9733-e0e295130165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:38.743507 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:38.803237 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:38.842057 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:38.843614 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:38.952228 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c73d2-0549-4d0a-9ddd-9769e81a0300 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:38.991013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:39.028022 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095c285-e634-445f-8f39-262580890740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:39.028676 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095c285-e634-445f-8f39-262580890740 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:39.028967 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095c285-e634-445f-8f39-262580890740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:39.029250 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095c285-e634-445f-8f39-262580890740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:39.030955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9095c285-e634-445f-8f39-262580890740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:39.141573 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095c285-e634-445f-8f39-262580890740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:39.143047 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095c285-e634-445f-8f39-262580890740 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:39.162039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 730/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:41.197787 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:41.198302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:41.198302 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:41.199648 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:41.205009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa89a-c89a-4680-921c-0ece6f9df0b8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:41.205498 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa89a-c89a-4680-921c-0ece6f9df0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:41.205598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa89a-c89a-4680-921c-0ece6f9df0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:41.280139 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d4ffa89a-c89a-4680-921c-0ece6f9df0b8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:41.281500 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d4ffa89a-c89a-4680-921c-0ece6f9df0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:41.302684 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:41.386665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:41.423949 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:41.425881 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:41.532828 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64601c-460e-4127-bf40-ddb5681f4e96 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:41.604001 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:41.611514 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:41.612172 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:41.612457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:41.612735 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:41.614458 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:41.723827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:41.725298 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8578f-08f4-4647-8565-e1ec043a1e22 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:41.744352 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 736/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:43.929053 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.929705 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.929705 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.931634 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:01:43.937491 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163386f-5bbb-4808-b899-84faa917c2e6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.938068 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163386f-5bbb-4808-b899-84faa917c2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.938173 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163386f-5bbb-4808-b899-84faa917c2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.006823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d163386f-5bbb-4808-b899-84faa917c2e6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:44.008055 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d163386f-5bbb-4808-b899-84faa917c2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:44.057010 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:44.105457 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:44.142959 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:44.144547 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:44.267154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d994d69-60e1-424a-8aff-152be0da1099 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.319541 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:44.346906 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.347568 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.347857 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.348137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.350433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:44.463839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.465429 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53095-6a3a-4f77-830d-33fc35cebb5c service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.485111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 733/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:54.103627 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=22 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:54.482721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=66 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:04.346931 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0b41-090b-42ea-be18-a0711150b47f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:04.347628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0b41-090b-42ea-be18-a0711150b47f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:04.347628 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0b41-090b-42ea-be18-a0711150b47f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:04.349424 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 738/1471] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:04.355102 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d0b-491a-4dd5-873b-29aa8e7c2788 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:04.355611 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d0b-491a-4dd5-873b-29aa8e7c2788 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:04.355721 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d0b-491a-4dd5-873b-29aa8e7c2788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:04.423225 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5fdc7d0b-491a-4dd5-873b-29aa8e7c2788 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:04.424348 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5fdc7d0b-491a-4dd5-873b-29aa8e7c2788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:04.524338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 734/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:04.616767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:04.617057 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:04.617340 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:04.618826 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 739/1473] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 11 12:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:04.624279 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1af7-51fb-42bf-b2a1-d71700e2dde3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:04.624575 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1af7-51fb-42bf-b2a1-d71700e2dde3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:04.624860 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1af7-51fb-42bf-b2a1-d71700e2dde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:04.709550 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-035d1af7-51fb-42bf-b2a1-d71700e2dde3 None None] MFA Rules not processed for user `55a5560632ed452c8287a7c91b57e420`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:04.711009 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-035d1af7-51fb-42bf-b2a1-d71700e2dde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:04.735162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:04.912027 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 735/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 11 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:04.919433 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:04.920963 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:05.025272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdb39-698b-4182-9be5-67796a09dbec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:05.100309 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55a5560632ed452c8287a7c91b57e420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:05.100966 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:05.101249 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:05.101525 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:05.103216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:05.158357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:05.215027 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:05.216492 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e334aa-75fe-4fe5-a43f-cde0b46f928d service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:05.236246 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 740/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:02:07.368538 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:07.369264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:07.369264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:07.370490 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 736/1476] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:07.376035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2654b3-e4f5-4fc9-904b-c0f7c3dce50d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:07.376513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2654b3-e4f5-4fc9-904b-c0f7c3dce50d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:07.376691 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2654b3-e4f5-4fc9-904b-c0f7c3dce50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:07.450940 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7a2654b3-e4f5-4fc9-904b-c0f7c3dce50d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:07.452186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7a2654b3-e4f5-4fc9-904b-c0f7c3dce50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:07.557402 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 741/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:07.580597 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:07.582148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:07.709266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a64e8-48c9-492b-b7c8-ec2981746a9c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:07.758091 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:07.803465 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:07.804133 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:07.804419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:07.804696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:07.805418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:07.807288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:02:07.807288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jul 11 12:02:07.807288 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: Jul 11 12:02:07.808419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d241aea-3a6b-4529-97cb-37c68fe3c175 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:07.827666 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 737/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 11 12:02:07 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:02:09.881211 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:09.881767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:09.881865 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:09.883288 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 742/1479] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:09.888808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b95-de3d-405e-8c77-42c533664169 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:09.889284 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b95-de3d-405e-8c77-42c533664169 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:09.889386 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b95-de3d-405e-8c77-42c533664169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:09.966670 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-046b5b95-de3d-405e-8c77-42c533664169 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:09.967889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-046b5b95-de3d-405e-8c77-42c533664169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:10.064806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 738/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:10.110020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:10.111608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:10.122210 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:10.272103 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fc95a-d7fe-4ffd-92e7-20f48b3168e8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:10.353344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:10.389164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:10.389788 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:10.390082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:10.390365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:10.392029 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:10.482991 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:10.484611 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d43a8-a31d-4412-9c90-1aab5f966fba service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:10.503549 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 743/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:12.730772 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:12.731359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:12.731359 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:12.732778 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 739/1482] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:12.738297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46280-65ed-48ab-90b4-da8ad3c19d7c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:12.738887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46280-65ed-48ab-90b4-da8ad3c19d7c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:12.738887 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef46280-65ed-48ab-90b4-da8ad3c19d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:12.803741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1ef46280-65ed-48ab-90b4-da8ad3c19d7c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:12.804964 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1ef46280-65ed-48ab-90b4-da8ad3c19d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:12.863385 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:12.921181 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 744/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:12.961696 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:12.963313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:12.988499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:13.070378 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05742b25-9707-433b-bef5-6e4b661f11ee None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:13.146287 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:13.146971 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:13.147253 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:13.147528 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:13.149173 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:13.223976 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:13.261667 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:13.263146 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ba94d-ebcf-44bf-a5cc-dd5dc181e5a6 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:13.281956 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 740/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:15.392852 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.393449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.393449 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.394861 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 745/1485] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:15.400488 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e8ec-a4f3-4e94-9b63-aab64d63943d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.400995 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e8ec-a4f3-4e94-9b63-aab64d63943d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.401096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e8ec-a4f3-4e94-9b63-aab64d63943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.472432 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1506e8ec-a4f3-4e94-9b63-aab64d63943d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:15.474252 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1506e8ec-a4f3-4e94-9b63-aab64d63943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.576536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 741/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:15.616278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.617853 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.728378 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e6cfe-b7a5-4982-9b68-16c8889d365f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.807637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:15.810272 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.811164 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.811668 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.811957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.815105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.942712 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.944075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af16d-56df-472b-a68a-e896021df3b0 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.962897 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 746/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:18.093210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:18.093806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:18.093806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:18.095187 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 742/1488] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:18.100614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca4d9-72c4-4c45-97e7-4daa74cd0f63 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:18.101144 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca4d9-72c4-4c45-97e7-4daa74cd0f63 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:18.101251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca4d9-72c4-4c45-97e7-4daa74cd0f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:18.172369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cb7ca4d9-72c4-4c45-97e7-4daa74cd0f63 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:18.173592 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cb7ca4d9-72c4-4c45-97e7-4daa74cd0f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:18.229975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:18.275758 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 747/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:18.315891 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:18.318052 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:18.358814 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:18.431123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7605cf-33d8-42b9-8760-2f8c6c47411a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:18.507866 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:18.508517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:18.508801 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:18.509078 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:18.510764 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:18.595298 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:18.626659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:18.628063 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1dda2-6350-4079-823a-edfc685260b7 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:18.647135 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 743/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:21.003589 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:21.004170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:21.004170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:21.005604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 748/1491] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 11 12:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:02:21.012980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6271c3-a378-4dfe-a490-993a0009454d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:21.013851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6271c3-a378-4dfe-a490-993a0009454d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:21.013851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6271c3-a378-4dfe-a490-993a0009454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:21.081763 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0f6271c3-a378-4dfe-a490-993a0009454d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:21.083543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0f6271c3-a378-4dfe-a490-993a0009454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:21.186566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 744/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 11 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:21.226268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:21.227871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:21.482904 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f1f88-a9eb-476c-84d9-cdc95d0d0e77 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:21.558658 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:21.559311 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:21.559593 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:21.559867 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:21.561511 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:21.570907 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:21.675258 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:21.677230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dccf8c-6db6-447f-9b61-7369f525a359 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:21.696075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 749/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:31.184658 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=82 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:31.693641 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=50 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:48.554527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:48.555137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:48.555137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:48.634350 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:48.635562 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:48.736527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 745/1494] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 11 12:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:48.745390 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:48.746026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:48.746026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:48.826696 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:48.827823 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:48.934257 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 750/1495] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 11 12:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:48.939289 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:48.940501 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:48.952367 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fce7e-c36f-40e7-b88b-f8f5349ba8af None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.015876 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:49.052675 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.053349 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.053631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.053938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.054664 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:03:49.054974 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.055991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.055991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jul 11 12:03:49.055991 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: Jul 11 12:03:49.056885 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.069024 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 746/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 11 12:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:03:49.077333 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:49.087051 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de0432-a18a-4ad0-b4ca-1af11b07cf49 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.153134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:49.189361 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.190066 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.190353 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.190666 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.191368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.193129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.193129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jul 11 12:03:49.193129 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: Jul 11 12:03:49.194928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.258251 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 751/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 11 12:03:49 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:03:49.262615 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:49.264167 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:49.302035 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735713-cb54-4160-bc47-ab7702a50043 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.416226 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.416852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.417140 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.417425 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.418181 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.419217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.419217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jul 11 12:03:49.419217 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: Jul 11 12:03:49.420464 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.464125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 747/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 11 12:03:49 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:03:49.467913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:49.469499 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:49.537494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293065e-9470-48c2-bf9e-285e757d72a8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.624973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:49.637515 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.638240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.638522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.638795 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.639557 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.640650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.640650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jul 11 12:03:49.640650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: Jul 11 12:03:49.641559 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.654358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 752/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 11 12:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:03:49.658405 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:49.660007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:49.671459 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdad18-0e26-4ef6-9097-5814958c51f2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.697116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:49.769518 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.770231 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.770519 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.770797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.771567 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.772586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.772586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jul 11 12:03:49.772586 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: Jul 11 12:03:49.773509 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05961d9d-0ea5-4771-a72a-a34f21165f30 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.786859 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 748/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 11 12:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:03:49.791186 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:49.792752 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:49.830669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d560318-f9b6-419f-9a26-01d2fbd6c11c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:49.926906 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:49.927717 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.928034 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.928310 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] PATH_INFO: `/v3/projects/1e2dff9b41634ad5ba49b394b64c97b7/users/9bbcb125940c4db591988076cdb3eded/roles/4971f34fbc684f48b7379abfd73b60f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:49.950608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2dff9b41634ad5ba49b394b64c97b7, user_id=9bbcb125940c4db591988076cdb3eded, role_id=4971f34fbc684f48b7379abfd73b60f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:49.951669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:49.951669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jul 11 12:03:49.951669 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: Jul 11 12:03:49.953642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349acaa-6994-44af-82f1-c3c319bf985f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:49.978783 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 753/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 11 12:03:49 2022] PUT /identity/v3/projects/1e2dff9b41634ad5ba49b394b64c97b7/users/9bbcb125940c4db591988076cdb3eded/roles/4971f34fbc684f48b7379abfd73b60f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:03:49.991401 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc8cc6-4848-4cd1-ac03-0a8359444b36 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:49.991701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc8cc6-4848-4cd1-ac03-0a8359444b36 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:49.991988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc8cc6-4848-4cd1-ac03-0a8359444b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.051767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-06dc8cc6-4848-4cd1-ac03-0a8359444b36 None None] MFA Rules not processed for user `9bbcb125940c4db591988076cdb3eded`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:50.052982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-06dc8cc6-4848-4cd1-ac03-0a8359444b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:50.136548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 749/1502] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 11 12:03:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:50.149643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:50.149976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:50.150281 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.151689 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 754/1503] 10.222.0.57 () {54 vars in 963 bytes} [Mon Jul 11 12:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:03:50.157071 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb294-8f1f-4e73-869d-a160acf68b66 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:50.157419 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb294-8f1f-4e73-869d-a160acf68b66 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:50.157774 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb294-8f1f-4e73-869d-a160acf68b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.188344 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:50.234472 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4b0fb294-8f1f-4e73-869d-a160acf68b66 None None] MFA Rules not processed for user `0cf7137aab9d4ebb8b83677cc4b0d817`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:50.235692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4b0fb294-8f1f-4e73-869d-a160acf68b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:50.311504 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 750/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Jul 11 12:03:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:50.319471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:50.321297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:50.372087 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:50.428858 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f36bc-fbcc-4cfb-801b-fe1cd5936e78 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:50.501875 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:50.502530 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:50.502815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:50.503130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.504767 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:50.608604 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:50.610105 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:50.613966 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:50.630516 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 755/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:03:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:03:50.674229 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1a1e-f403-4f4a-ad1c-e79c760ae6f4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:50.674646 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1a1e-f403-4f4a-ad1c-e79c760ae6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:50.674800 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a1a1e-f403-4f4a-ad1c-e79c760ae6f4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.676084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 751/1506] 10.222.0.57 () {52 vars in 829 bytes} [Mon Jul 11 12:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:03:50.679770 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:50.680988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:50.780044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf249-d8d3-4eaf-8846-7a54fd2d535d service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:50.879154 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:50.880072 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:50.880358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:50.880634 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] PATH_INFO: `/v3/users/9bbcb125940c4db591988076cdb3eded` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:50.893188 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbcb125940c4db591988076cdb3eded)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:50.894254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:50.894254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jul 11 12:03:50.894254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: Jul 11 12:03:50.895472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:50.905622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:50.961809 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbcb125940c4db591988076cdb3eded'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:50.962243 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbcb125940c4db591988076cdb3eded'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:50.982880 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbcb125940c4db591988076cdb3eded'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:50.992454 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbcb125940c4db591988076cdb3eded'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:51.004782 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbcb125940c4db591988076cdb3eded'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:51.010411 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 756/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 11 12:03:50 2022] DELETE /identity/v3/users/9bbcb125940c4db591988076cdb3eded => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:03:51.014923 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:51.015213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:51.015491 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:51.071193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:51.072320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:51.150079 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:51.163665 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 752/1508] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 11 12:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:51.173263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:51.176279 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:51.275166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a7e9-addf-43ec-a292-16bea5244c61 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:51.332569 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:51.354242 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:51.354901 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:51.355189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:51.355469 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:51.357107 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:51.471550 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:51.473039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055bf6e-7d2b-4e89-9280-7701ad933b24 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:51.491192 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 757/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:03:51.849093 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:51.850844 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:51.864830 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc84a4-8ec6-4915-9154-ec96ce58df8c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:51.978765 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4c463b509f1490f91e3e253190d2a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aeb90a4a47b148ccb3710178a3b17750', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:51.979659 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:51.979945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:51.980220 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] PATH_INFO: `/v3/projects/1e2dff9b41634ad5ba49b394b64c97b7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:51.990686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2dff9b41634ad5ba49b394b64c97b7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:51.991709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 12:03:51.991709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jul 11 12:03:51.991709 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: Jul 11 12:03:51.992913 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:52.051839 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2dff9b41634ad5ba49b394b64c97b7', 'request_id': 'req-118969b2-3e75-487f-be0e-d5c6ad07006b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:52.057911 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2dff9b41634ad5ba49b394b64c97b7', 'request_id': 'req-118969b2-3e75-487f-be0e-d5c6ad07006b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:52.071728 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2dff9b41634ad5ba49b394b64c97b7', 'request_id': 'req-118969b2-3e75-487f-be0e-d5c6ad07006b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:52.098387 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] The token cache is being invalidate because project 1e2dff9b41634ad5ba49b394b64c97b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:03:52.098588 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:52.099114 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-118969b2-3e75-487f-be0e-d5c6ad07006b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:03:52.100517 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 753/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 11 12:03:51 2022] DELETE /identity/v3/projects/1e2dff9b41634ad5ba49b394b64c97b7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:04:01.488994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=62 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:04:02.096155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:37.008815 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:37.009419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:37.009419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:37.010791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:37.015921 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf62b-cc31-4123-85fa-be69423520f1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:37.016366 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf62b-cc31-4123-85fa-be69423520f1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:37.016471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf62b-cc31-4123-85fa-be69423520f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:37.096932 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5bbcf62b-cc31-4123-85fa-be69423520f1 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:37.098701 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5bbcf62b-cc31-4123-85fa-be69423520f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:37.201038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:37.356741 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:37.359020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:37.475387 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1243e-5ab6-421c-8846-63e3ab91692d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:37.480873 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:37.481513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:37.481831 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:37.482111 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:37.491607 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:37.493200 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ed3c2-6265-4eaf-9c46-758ba5ff6068 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:37.502730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 759/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:38.322111 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf24ab-6153-4f46-ba7a-1952e72130f9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:38.322711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf24ab-6153-4f46-ba7a-1952e72130f9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:38.322711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf24ab-6153-4f46-ba7a-1952e72130f9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:38.324147 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:05:38.329383 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e18f1b-f442-4fb7-8172-40b0b5af8143 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:38.329899 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e18f1b-f442-4fb7-8172-40b0b5af8143 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:38.330020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e18f1b-f442-4fb7-8172-40b0b5af8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:38.349799 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d7e18f1b-f442-4fb7-8172-40b0b5af8143 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:38.351170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d7e18f1b-f442-4fb7-8172-40b0b5af8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:38.358938 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:38.508770 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf24ab-6153-4f46-ba7a-1952e72130f9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:38.523270 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf24ab-6153-4f46-ba7a-1952e72130f9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:38.528711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:38.529347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:38.529635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:38.529943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:38.538552 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:38.540138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c66f3-8c60-4faf-87aa-7e52d6aad228 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:38.549545 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 756/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:39.060077 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f379e1-1c26-4cd2-9a61-70a7ed183976 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:39.060363 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f379e1-1c26-4cd2-9a61-70a7ed183976 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:39.060643 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f379e1-1c26-4cd2-9a61-70a7ed183976 None None] PATH_INFO: `/v3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:39.062041 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 761/1517] 10.222.0.27 () {54 vars in 925 bytes} [Mon Jul 11 12:05:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:05:39.069096 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019df14-a760-431f-b4db-d6fc5fdac0dc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:39.069395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019df14-a760-431f-b4db-d6fc5fdac0dc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:39.069694 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019df14-a760-431f-b4db-d6fc5fdac0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:39.128254 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2019df14-a760-431f-b4db-d6fc5fdac0dc None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:39.129527 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2019df14-a760-431f-b4db-d6fc5fdac0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:39.146844 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:39.204210 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 757/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Jul 11 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:48.344988 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=56 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:48.552388 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee565-fbd5-435e-b591-28b3b1c719d2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:48.552675 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee565-fbd5-435e-b591-28b3b1c719d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:48.552957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee565-fbd5-435e-b591-28b3b1c719d2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:48.554394 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 762/1519] 10.222.0.57 () {54 vars in 960 bytes} [Mon Jul 11 12:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:48.559662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04458941-f324-483f-b899-e998583753b9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:48.560153 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04458941-f324-483f-b899-e998583753b9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:48.560263 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04458941-f324-483f-b899-e998583753b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:48.580444 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-04458941-f324-483f-b899-e998583753b9 None None] MFA Rules not processed for user `a6db07035b4d472ebc58509502b666ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:48.581655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-04458941-f324-483f-b899-e998583753b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:48.591192 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 758/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Mon Jul 11 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:48.599060 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee565-fbd5-435e-b591-28b3b1c719d2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:48.612325 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee565-fbd5-435e-b591-28b3b1c719d2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:48.617704 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6db07035b4d472ebc58509502b666ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:48.618351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:48.618635 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:48.618908 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:48.627487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:48.629085 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5e3ba-a88f-4006-baff-2532a161ef42 service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:48.639740 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 763/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:49.784437 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e784d49-c9bc-4708-b0f0-d18882ccf3c8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:49.784996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e784d49-c9bc-4708-b0f0-d18882ccf3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:49.784996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e784d49-c9bc-4708-b0f0-d18882ccf3c8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:49.786931 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:49.791700 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325122-6292-4cbe-b56b-15778e2032fa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:49.792148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325122-6292-4cbe-b56b-15778e2032fa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:49.792254 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325122-6292-4cbe-b56b-15778e2032fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:49.810870 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-83325122-6292-4cbe-b56b-15778e2032fa None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:49.812061 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-83325122-6292-4cbe-b56b-15778e2032fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:49.819872 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:05:49.974189 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e784d49-c9bc-4708-b0f0-d18882ccf3c8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:49.988628 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e784d49-c9bc-4708-b0f0-d18882ccf3c8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:49.995199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:49.995841 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:49.996131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:49.996414 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:50.004829 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:50.006474 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbe338-eeca-46d0-9a8a-df91d3993fe2 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:50.016982 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 760/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:05:50.845461 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5caa2-656b-426d-9a9e-099e9beddeb2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:50.846135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5caa2-656b-426d-9a9e-099e9beddeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:50.846135 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5caa2-656b-426d-9a9e-099e9beddeb2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:50.847500 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:50.852286 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e797-d5f6-42a2-89e0-cce27e2be16a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:50.852741 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e797-d5f6-42a2-89e0-cce27e2be16a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:50.852855 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e797-d5f6-42a2-89e0-cce27e2be16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:50.870833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7e93e797-d5f6-42a2-89e0-cce27e2be16a None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:50.872013 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7e93e797-d5f6-42a2-89e0-cce27e2be16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:50.880525 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:51.030204 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5caa2-656b-426d-9a9e-099e9beddeb2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:51.045145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5caa2-656b-426d-9a9e-099e9beddeb2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:51.050526 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80890a6017284bf090892fbcbb618f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:51.051167 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:51.051462 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:51.051744 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:51.059837 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:51.061416 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa249f1-ab3c-4c44-bf8b-51c6c5c3366b service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:51.070846 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 766/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:55.144452 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483fea-11db-4125-adb5-2415158680d5 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:55.145004 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483fea-11db-4125-adb5-2415158680d5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:55.145635 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483fea-11db-4125-adb5-2415158680d5 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:55.147138 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:55.152474 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41fd23-cbe0-4c95-b461-7f6ed3a6c664 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:55.152925 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41fd23-cbe0-4c95-b461-7f6ed3a6c664 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:55.153039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41fd23-cbe0-4c95-b461-7f6ed3a6c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:55.170776 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9d41fd23-cbe0-4c95-b461-7f6ed3a6c664 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:55.171985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9d41fd23-cbe0-4c95-b461-7f6ed3a6c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:55.179913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:55.255560 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483fea-11db-4125-adb5-2415158680d5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:55.257274 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-de483fea-11db-4125-adb5-2415158680d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:55.336273 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483fea-11db-4125-adb5-2415158680d5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:55.340206 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:55.340795 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:55.341081 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:55.341363 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:55.350479 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:55.351916 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d39de3-279f-4e9e-93e4-4204f5d443bd service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:55.361251 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 763/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:57.380822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a9ab-7716-48ed-9790-c55cc32cd4e7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:57.381359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a9ab-7716-48ed-9790-c55cc32cd4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:57.381359 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a9ab-7716-48ed-9790-c55cc32cd4e7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:57.382798 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:05:57.388107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687a0b8-2a03-4495-92ce-b3e505342d78 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:57.388556 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687a0b8-2a03-4495-92ce-b3e505342d78 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:57.388661 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687a0b8-2a03-4495-92ce-b3e505342d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:57.406927 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7687a0b8-2a03-4495-92ce-b3e505342d78 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:57.408115 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7687a0b8-2a03-4495-92ce-b3e505342d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:57.415955 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:57.450818 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06a9ab-7716-48ed-9790-c55cc32cd4e7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:57.462760 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06a9ab-7716-48ed-9790-c55cc32cd4e7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:57.466558 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5a31865c8ed4ab78bfdb17780b61d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:57.467189 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:57.467473 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:57.467749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:57.476619 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:57.478101 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fada7b0-bbc1-4586-8d80-cf179d6ee7e0 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:57.487541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 769/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:01.064692 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:01.065275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:01.065275 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:01.066719 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:01.072132 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129cb7-98cc-4149-a649-78446b4062a6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:01.072632 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129cb7-98cc-4149-a649-78446b4062a6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:01.072749 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129cb7-98cc-4149-a649-78446b4062a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:01.090766 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-08129cb7-98cc-4149-a649-78446b4062a6 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:01.091984 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-08129cb7-98cc-4149-a649-78446b4062a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:01.100503 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:01.408014 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:01.409690 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:01.477396 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:01.495900 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94948624-96a7-4cc8-b306-6c3941a22271 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:01.499662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:01.500320 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:01.500610 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:01.500895 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:01.510700 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:01.512154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ee18c-6bc4-49bd-b372-a6c0651596cd service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:01.523318 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 766/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:03.591537 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca923971-d104-4faa-96fb-7727da154968 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:03.592114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca923971-d104-4faa-96fb-7727da154968 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:03.592114 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca923971-d104-4faa-96fb-7727da154968 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:03.593485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:03.598899 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c349-d549-4aae-95cb-3aab510a7d20 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:03.599384 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c349-d549-4aae-95cb-3aab510a7d20 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:03.599492 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7c349-d549-4aae-95cb-3aab510a7d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:03.617889 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e2f7c349-d549-4aae-95cb-3aab510a7d20 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:03.619264 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e2f7c349-d549-4aae-95cb-3aab510a7d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:03.627410 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:03.932851 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca923971-d104-4faa-96fb-7727da154968 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:03.934789 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ca923971-d104-4faa-96fb-7727da154968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:03.949067 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca923971-d104-4faa-96fb-7727da154968 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:03.953165 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:03.953637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:03.953961 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:03.954240 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:03.964451 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:03.965923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24c213-308b-464a-87ab-2e79b95e6a4d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:03.976021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 772/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:13.612420 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=11 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:13.971344 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=33 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:16.452618 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253959b1-43c9-405e-9ffa-c8b0c1e675aa None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:16.453132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253959b1-43c9-405e-9ffa-c8b0c1e675aa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:16.453132 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253959b1-43c9-405e-9ffa-c8b0c1e675aa None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:16.454532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 768/1540] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:16.460301 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1dab7-bc04-46a4-a14c-4bbeb0007033 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:16.460822 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1dab7-bc04-46a4-a14c-4bbeb0007033 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:16.460920 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1dab7-bc04-46a4-a14c-4bbeb0007033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:16.479278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6cd1dab7-bc04-46a4-a14c-4bbeb0007033 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:16.480522 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6cd1dab7-bc04-46a4-a14c-4bbeb0007033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:16.488296 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 773/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:16.790353 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253959b1-43c9-405e-9ffa-c8b0c1e675aa None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:16.800930 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253959b1-43c9-405e-9ffa-c8b0c1e675aa None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:16.804406 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:16.805038 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:16.805323 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:16.805603 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:16.814484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:16.815942 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c8f3e-eecf-4d63-913d-8457da5a477c service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:16.825811 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 769/1542] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:18.871232 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f181c5-2358-40df-b7b1-114fe44be298 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:18.871778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f181c5-2358-40df-b7b1-114fe44be298 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:18.871778 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f181c5-2358-40df-b7b1-114fe44be298 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:18.873170 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 774/1543] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:18.878534 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39651bb4-f396-4af4-a009-08a015e3d3ae None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:18.879542 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39651bb4-f396-4af4-a009-08a015e3d3ae None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:18.879732 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39651bb4-f396-4af4-a009-08a015e3d3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:18.897898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-39651bb4-f396-4af4-a009-08a015e3d3ae None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:18.899098 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-39651bb4-f396-4af4-a009-08a015e3d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:18.907031 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 770/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:19.207597 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f181c5-2358-40df-b7b1-114fe44be298 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:19.218551 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f181c5-2358-40df-b7b1-114fe44be298 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:19.222075 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:19.223259 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:19.223545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:19.223827 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:19.232685 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:19.234148 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61542db5-1673-480f-bdd5-b72eb3a71cff service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:19.244069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 775/1545] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:28.892370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:29.240039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:31.716347 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044db4b9-fdbd-45f6-9d58-b494cad80e70 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:31.716943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044db4b9-fdbd-45f6-9d58-b494cad80e70 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:31.716943 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044db4b9-fdbd-45f6-9d58-b494cad80e70 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:31.718357 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 771/1546] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 12:06:31.723488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b23dc-18d2-4d7a-9236-41a874e00291 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:31.723909 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b23dc-18d2-4d7a-9236-41a874e00291 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:31.724046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b23dc-18d2-4d7a-9236-41a874e00291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:31.741637 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c58b23dc-18d2-4d7a-9236-41a874e00291 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:31.742855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c58b23dc-18d2-4d7a-9236-41a874e00291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:31.750608 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 776/1547] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:32.049130 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044db4b9-fdbd-45f6-9d58-b494cad80e70 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:32.060186 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044db4b9-fdbd-45f6-9d58-b494cad80e70 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:32.063680 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:32.064311 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:32.064594 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:32.064869 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:32.074308 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:32.075730 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e19d58-310f-4ec5-9ed7-96aed11f7bc3 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:32.085029 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 772/1548] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:34.150351 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36027-692a-48fd-aa75-1e6d7d702e34 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:34.150894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36027-692a-48fd-aa75-1e6d7d702e34 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:34.150894 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36027-692a-48fd-aa75-1e6d7d702e34 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:34.152284 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 777/1549] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:34.157337 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693cb84-257c-4fcd-8f3b-e4114a139b87 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:34.157802 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693cb84-257c-4fcd-8f3b-e4114a139b87 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:34.157939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693cb84-257c-4fcd-8f3b-e4114a139b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:34.176980 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7693cb84-257c-4fcd-8f3b-e4114a139b87 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:34.178213 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7693cb84-257c-4fcd-8f3b-e4114a139b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:34.186299 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 773/1550] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:34.490541 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36027-692a-48fd-aa75-1e6d7d702e34 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:34.501401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36027-692a-48fd-aa75-1e6d7d702e34 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:34.504970 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:34.505638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:34.505957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:34.506241 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:34.515209 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:34.516642 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4edd9-f6a8-4633-a206-efb06344d33b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:34.526650 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 778/1551] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:44.171277 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:44.522448 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:47.014807 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2679-af8a-4820-9396-915f7139cd43 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:47.015411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2679-af8a-4820-9396-915f7139cd43 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:47.015411 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2679-af8a-4820-9396-915f7139cd43 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:47.016806 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 774/1552] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:47.022638 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987dbf-717f-4d52-9db7-ff5d1260c68e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:47.023052 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987dbf-717f-4d52-9db7-ff5d1260c68e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:47.023196 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38987dbf-717f-4d52-9db7-ff5d1260c68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:47.040973 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-38987dbf-717f-4d52-9db7-ff5d1260c68e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:47.042214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-38987dbf-717f-4d52-9db7-ff5d1260c68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:47.050276 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 779/1553] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:47.355945 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c2679-af8a-4820-9396-915f7139cd43 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:47.366536 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c2679-af8a-4820-9396-915f7139cd43 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:47.370007 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:47.370638 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:47.370921 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:47.371205 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:47.379851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:47.381266 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e3df7-aa95-466c-b0e4-26e6b5be1f9a service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:47.390418 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 775/1554] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:49.444443 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaecf9-6f2f-4f37-8267-01330e610159 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:49.444953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaecf9-6f2f-4f37-8267-01330e610159 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:49.444953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaecf9-6f2f-4f37-8267-01330e610159 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:49.446369 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 780/1555] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:06:49.451751 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47e32c-0b5f-4257-b996-384dc6e43996 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:49.452145 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47e32c-0b5f-4257-b996-384dc6e43996 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:49.452352 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47e32c-0b5f-4257-b996-384dc6e43996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:49.470175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ba47e32c-0b5f-4257-b996-384dc6e43996 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:06:49.471371 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ba47e32c-0b5f-4257-b996-384dc6e43996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:06:49.479177 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 776/1556] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:06:49.781483 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eaecf9-6f2f-4f37-8267-01330e610159 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:06:49.792838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eaecf9-6f2f-4f37-8267-01330e610159 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:06:49.796381 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:06:49.797019 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:06:49.797312 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:06:49.797623 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:06:49.807086 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:06:49.808521 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a3f8b-8654-4625-be26-26ee690bac5a service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:06:49.818171 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 781/1557] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:59.465113 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:06:59.813913 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:02.288703 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8032f-4be4-48c9-8812-1393e2ab8b22 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:02.289283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8032f-4be4-48c9-8812-1393e2ab8b22 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:02.289283 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8032f-4be4-48c9-8812-1393e2ab8b22 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:02.290686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 777/1558] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:02.295838 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c35799-d5dc-4f1f-b654-ad2b9e908919 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:02.296266 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c35799-d5dc-4f1f-b654-ad2b9e908919 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:02.296401 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c35799-d5dc-4f1f-b654-ad2b9e908919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:02.314804 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b3c35799-d5dc-4f1f-b654-ad2b9e908919 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:02.316007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b3c35799-d5dc-4f1f-b654-ad2b9e908919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:02.324690 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 782/1559] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:02.622938 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8032f-4be4-48c9-8812-1393e2ab8b22 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:02.633471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8032f-4be4-48c9-8812-1393e2ab8b22 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:02.636962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:02.637605 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:02.637918 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:02.638199 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:02.648276 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:02.649767 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74dd20-7f82-4e3b-982e-07db5260788a service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:02.660933 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 778/1560] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:04.711084 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6f3aa-6b1d-456e-9959-6907421df688 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:04.711663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6f3aa-6b1d-456e-9959-6907421df688 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:04.711663 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6f3aa-6b1d-456e-9959-6907421df688 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:04.713080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 783/1561] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:04.720336 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c859-817a-47cb-8c28-8ac7dfb62225 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:04.720867 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c859-817a-47cb-8c28-8ac7dfb62225 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:04.721045 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c859-817a-47cb-8c28-8ac7dfb62225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:04.740059 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c5c6c859-817a-47cb-8c28-8ac7dfb62225 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:04.741268 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5c6c859-817a-47cb-8c28-8ac7dfb62225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:04.749321 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 779/1562] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:05.053879 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6f3aa-6b1d-456e-9959-6907421df688 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:05.064580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6f3aa-6b1d-456e-9959-6907421df688 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:05.068112 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:05.068750 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:05.069039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:05.069318 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:05.078216 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:05.079651 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160159-3272-412f-a237-d18b5fb1537b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:05.091560 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 784/1563] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:14.734300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:15.085357 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:17.601499 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b5f2-6fee-4b67-a880-69583ee9032f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:17.602123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b5f2-6fee-4b67-a880-69583ee9032f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:17.602123 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b5f2-6fee-4b67-a880-69583ee9032f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:17.603537 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 780/1564] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:17.608587 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d8572-82e6-4842-96d3-21ec3cab481e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:17.609047 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d8572-82e6-4842-96d3-21ec3cab481e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:17.609166 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d8572-82e6-4842-96d3-21ec3cab481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:17.626958 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2b5d8572-82e6-4842-96d3-21ec3cab481e None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:17.628145 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2b5d8572-82e6-4842-96d3-21ec3cab481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:17.636482 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 785/1565] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:17.932834 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b5f2-6fee-4b67-a880-69583ee9032f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:17.944107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b5f2-6fee-4b67-a880-69583ee9032f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:17.947566 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:17.948203 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:17.948485 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:17.948760 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:17.957548 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:17.959001 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbf7fd-eea3-4d2e-8041-5edfc53320e6 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:17.968864 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 781/1566] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:20.039051 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1980d-955e-4580-89a0-b7a02a1287bf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:20.039622 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1980d-955e-4580-89a0-b7a02a1287bf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:20.039730 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1980d-955e-4580-89a0-b7a02a1287bf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:20.041037 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 786/1567] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:20.046577 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abb0-b7a0-4957-8e53-ac02bad360d8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:20.046996 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abb0-b7a0-4957-8e53-ac02bad360d8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:20.047146 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abb0-b7a0-4957-8e53-ac02bad360d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:20.065332 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-43e1abb0-b7a0-4957-8e53-ac02bad360d8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:20.066623 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-43e1abb0-b7a0-4957-8e53-ac02bad360d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:20.074655 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 782/1568] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:20.385983 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1980d-955e-4580-89a0-b7a02a1287bf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:20.397160 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1980d-955e-4580-89a0-b7a02a1287bf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:20.400760 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:20.401393 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:20.401702 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:20.401994 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:20.411545 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:20.412969 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a242ad0-b8a3-4c30-8708-2cde51ad3648 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:20.422395 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 787/1569] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:30.060328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:30.418331 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:32.875852 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dce5-ced5-43e5-9db5-1e79324995e9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:32.876397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dce5-ced5-43e5-9db5-1e79324995e9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:32.876397 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dce5-ced5-43e5-9db5-1e79324995e9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:32.878621 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 783/1570] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:32.883487 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62891d-cc3b-49b9-964e-df96ca36ed00 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:32.883892 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62891d-cc3b-49b9-964e-df96ca36ed00 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:32.884044 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62891d-cc3b-49b9-964e-df96ca36ed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:32.901494 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ca62891d-cc3b-49b9-964e-df96ca36ed00 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:32.902711 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ca62891d-cc3b-49b9-964e-df96ca36ed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:32.910345 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 788/1571] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:33.210633 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0dce5-ced5-43e5-9db5-1e79324995e9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:33.221461 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0dce5-ced5-43e5-9db5-1e79324995e9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:33.225543 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:33.226209 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:33.226494 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:33.226769 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:33.235579 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:33.237025 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821eb44-4ad2-4499-83d0-fc2b596e8e30 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:33.246898 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 784/1572] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:35.305896 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27686c3-a17f-4eef-b2ad-b74ed8fe7ade None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:35.306470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27686c3-a17f-4eef-b2ad-b74ed8fe7ade None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:35.306470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27686c3-a17f-4eef-b2ad-b74ed8fe7ade None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:35.307855 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 789/1573] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:35.313090 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57f378-24ec-4a9a-9319-3683ee098ca4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:35.313484 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57f378-24ec-4a9a-9319-3683ee098ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:35.313686 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57f378-24ec-4a9a-9319-3683ee098ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:35.331972 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dd57f378-24ec-4a9a-9319-3683ee098ca4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:35.333175 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dd57f378-24ec-4a9a-9319-3683ee098ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:35.341168 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 785/1574] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:35.641073 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27686c3-a17f-4eef-b2ad-b74ed8fe7ade None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:35.652370 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27686c3-a17f-4eef-b2ad-b74ed8fe7ade None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:35.655885 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:35.656517 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:35.656803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:35.657082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:35.666021 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:35.667485 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f985e-e298-42d1-85f5-301c9626e19e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:35.677131 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 790/1575] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:45.326395 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:45.672923 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:48.179239 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443cc94-ea9a-4a19-9b85-29c2aad60591 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:48.179797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443cc94-ea9a-4a19-9b85-29c2aad60591 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:48.179797 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443cc94-ea9a-4a19-9b85-29c2aad60591 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:48.181178 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 786/1576] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:48.186405 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9088-b1e2-4c9a-ac7e-6dbd7937fcf3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:48.186871 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9088-b1e2-4c9a-ac7e-6dbd7937fcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:48.186976 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f9088-b1e2-4c9a-ac7e-6dbd7937fcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:48.205382 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-637f9088-b1e2-4c9a-ac7e-6dbd7937fcf3 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:48.206616 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-637f9088-b1e2-4c9a-ac7e-6dbd7937fcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:48.214703 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 791/1577] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:48.515408 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443cc94-ea9a-4a19-9b85-29c2aad60591 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:48.526086 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443cc94-ea9a-4a19-9b85-29c2aad60591 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:48.529555 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:48.530219 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:48.530507 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:48.530788 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:48.540236 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:48.541657 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca08f9-e7e1-4345-aea6-81011814d578 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:48.553084 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 787/1578] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:50.598584 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa311b-218a-4711-a8f3-6fb9b22c42ad None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:50.599115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa311b-218a-4711-a8f3-6fb9b22c42ad None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:50.599115 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa311b-218a-4711-a8f3-6fb9b22c42ad None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:50.600426 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 792/1579] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:07:50.605559 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba98d2-8643-4435-8937-54ad0fd37f20 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:50.606328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba98d2-8643-4435-8937-54ad0fd37f20 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:50.606328 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba98d2-8643-4435-8937-54ad0fd37f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:50.624747 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-80ba98d2-8643-4435-8937-54ad0fd37f20 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:07:50.625962 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-80ba98d2-8643-4435-8937-54ad0fd37f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:50.634546 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 788/1580] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:07:50.931889 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa311b-218a-4711-a8f3-6fb9b22c42ad None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:50.943368 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa311b-218a-4711-a8f3-6fb9b22c42ad None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:50.946877 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:50.947513 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:50.947800 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:50.948080 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:50.957077 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:50.958538 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9125c-61cd-437d-a34b-a283ca06a055 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:50.968329 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:00.618786 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:00.964134 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:03.436532 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c9dd-794f-4d0c-9d50-0a89a33ae367 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:03.437148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c9dd-794f-4d0c-9d50-0a89a33ae367 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:03.437148 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c9dd-794f-4d0c-9d50-0a89a33ae367 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:03.438426 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 789/1582] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:03.444737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1747-8189-4a2f-9051-bdcf3c929bfb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:03.445178 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1747-8189-4a2f-9051-bdcf3c929bfb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:03.445472 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe1747-8189-4a2f-9051-bdcf3c929bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:03.464174 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-03fe1747-8189-4a2f-9051-bdcf3c929bfb None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:03.465364 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-03fe1747-8189-4a2f-9051-bdcf3c929bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:03.473380 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 794/1583] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:03.774892 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5c9dd-794f-4d0c-9d50-0a89a33ae367 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:03.788334 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5c9dd-794f-4d0c-9d50-0a89a33ae367 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:03.791888 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:03.792530 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:03.792818 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:03.793095 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:03.802706 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:03.804155 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e78926-349c-4675-a3d4-cfd54c526768 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:03.813508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 790/1584] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:05.879726 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cc90-f32a-47bc-b9c4-b11081175f4c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:05.880278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cc90-f32a-47bc-b9c4-b11081175f4c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:05.880278 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cc90-f32a-47bc-b9c4-b11081175f4c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:05.881567 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 795/1585] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:05.886734 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979da4-7e38-4a37-acc9-4fd1ba140d8d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:05.887150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979da4-7e38-4a37-acc9-4fd1ba140d8d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:05.887324 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979da4-7e38-4a37-acc9-4fd1ba140d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:05.905193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2e979da4-7e38-4a37-acc9-4fd1ba140d8d None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:05.906582 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2e979da4-7e38-4a37-acc9-4fd1ba140d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:05.914313 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 791/1586] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:06.214957 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0cc90-f32a-47bc-b9c4-b11081175f4c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:06.226268 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0cc90-f32a-47bc-b9c4-b11081175f4c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:06.229784 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:06.230421 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:06.230706 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:06.230985 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:06.240341 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:06.241791 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294a4a3-cea2-4052-8137-8360eb93e038 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:06.251070 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 796/1587] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:15.900116 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:16.246944 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:19.137182 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d596c-4ddc-44cb-844a-74893728cf8b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:19.137752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d596c-4ddc-44cb-844a-74893728cf8b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:19.137752 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d596c-4ddc-44cb-844a-74893728cf8b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:19.139149 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 792/1588] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:19.144365 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85d54-4040-4c70-b7b3-299746707bbb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:19.144805 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85d54-4040-4c70-b7b3-299746707bbb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:19.144932 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85d54-4040-4c70-b7b3-299746707bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:19.164306 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-12c85d54-4040-4c70-b7b3-299746707bbb None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:19.165496 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-12c85d54-4040-4c70-b7b3-299746707bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:19.173342 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 797/1589] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:08:19.483512 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d596c-4ddc-44cb-844a-74893728cf8b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:19.494804 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d596c-4ddc-44cb-844a-74893728cf8b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:19.498413 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d749a-c805-4803-a465-508528243679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:19.499055 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d749a-c805-4803-a465-508528243679 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:19.499343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d749a-c805-4803-a465-508528243679 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:19.499625 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d749a-c805-4803-a465-508528243679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:19.508632 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d749a-c805-4803-a465-508528243679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:19.510125 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d749a-c805-4803-a465-508528243679 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:19.523909 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 793/1590] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:21.759020 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523ae4b-611e-4afd-adfc-4e9427366836 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:21.759609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523ae4b-611e-4afd-adfc-4e9427366836 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:21.759609 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523ae4b-611e-4afd-adfc-4e9427366836 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:21.760975 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 798/1591] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:21.766039 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164a84-d5d7-44cf-8b53-412b4915cc6b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:21.766467 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164a84-d5d7-44cf-8b53-412b4915cc6b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:21.766598 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164a84-d5d7-44cf-8b53-412b4915cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:21.784150 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-59164a84-d5d7-44cf-8b53-412b4915cc6b None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:21.785338 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-59164a84-d5d7-44cf-8b53-412b4915cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:21.793342 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 794/1592] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:22.092891 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523ae4b-611e-4afd-adfc-4e9427366836 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:22.104218 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523ae4b-611e-4afd-adfc-4e9427366836 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:22.107736 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:22.108374 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:22.108661 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:22.108941 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:22.119007 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:22.120470 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c852e-aa90-4ad0-b647-f1fbc7d3a734 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:22.130580 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 799/1593] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:31.779162 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:32.126064 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:34.582716 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aba488-82aa-4b9a-87df-857d46db2092 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:34.583278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aba488-82aa-4b9a-87df-857d46db2092 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:34.583278 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aba488-82aa-4b9a-87df-857d46db2092 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:34.584643 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 795/1594] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:34.590130 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7380c-6906-425c-9c72-0bc184e92905 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:34.590572 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7380c-6906-425c-9c72-0bc184e92905 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:34.590737 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7380c-6906-425c-9c72-0bc184e92905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:34.609618 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f3d7380c-6906-425c-9c72-0bc184e92905 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:34.610208 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3d7380c-6906-425c-9c72-0bc184e92905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:34.617934 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 800/1595] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:34.919503 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aba488-82aa-4b9a-87df-857d46db2092 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:34.930018 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aba488-82aa-4b9a-87df-857d46db2092 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:34.933595 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:34.934257 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:34.934540 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:34.934816 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:34.944193 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:34.945601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d18a4-cf21-4773-95b3-fe22f156cd7b service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:34.954851 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 796/1596] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:37.037069 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009cd44-e7e2-4dd3-b288-fc7c7195fb94 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:37.037646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009cd44-e7e2-4dd3-b288-fc7c7195fb94 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:37.037646 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009cd44-e7e2-4dd3-b288-fc7c7195fb94 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:37.039039 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 801/1597] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:37.044164 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b60a3-3b2e-403b-8253-74f3fee2de54 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:37.044600 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b60a3-3b2e-403b-8253-74f3fee2de54 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:37.044710 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b60a3-3b2e-403b-8253-74f3fee2de54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:37.062958 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-617b60a3-3b2e-403b-8253-74f3fee2de54 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:37.064154 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-617b60a3-3b2e-403b-8253-74f3fee2de54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:37.071939 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 797/1598] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:37.381238 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009cd44-e7e2-4dd3-b288-fc7c7195fb94 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:37.392126 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009cd44-e7e2-4dd3-b288-fc7c7195fb94 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:37.396452 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:37.397086 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:37.397445 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:37.397655 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:37.406555 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:37.407977 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06238bc-fe4f-43cc-806c-aa69f0bc8265 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:37.418026 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 802/1599] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:47.057960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:47.414043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:49.924718 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3d63-4ff6-4426-a2dc-8e5193f0a0d2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:49.925315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3d63-4ff6-4426-a2dc-8e5193f0a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:49.925315 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3d63-4ff6-4426-a2dc-8e5193f0a0d2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:49.926707 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 798/1600] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:49.931928 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b931-3c0b-48b6-b15a-1e13a3037fdd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:49.932437 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b931-3c0b-48b6-b15a-1e13a3037fdd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:49.932550 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b931-3c0b-48b6-b15a-1e13a3037fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:49.951024 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cf21b931-3c0b-48b6-b15a-1e13a3037fdd None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:49.952234 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf21b931-3c0b-48b6-b15a-1e13a3037fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:49.960803 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 803/1601] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:08:50.262060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3d63-4ff6-4426-a2dc-8e5193f0a0d2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:50.272662 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3d63-4ff6-4426-a2dc-8e5193f0a0d2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:50.276139 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:50.276773 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:50.277060 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:50.277482 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:50.286833 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:50.288300 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa3153-9cc3-485c-ada1-22d4497243a2 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:50.297904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 799/1602] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:08:52.353854 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ea18-e5b0-4473-ac18-44df8d5021c8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:52.354419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ea18-e5b0-4473-ac18-44df8d5021c8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:52.354419 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ea18-e5b0-4473-ac18-44df8d5021c8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:52.355832 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 804/1603] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:08:52.360988 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2705-cf45-4ba0-a812-ab6e67966ef8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:52.361431 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2705-cf45-4ba0-a812-ab6e67966ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:52.361563 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2705-cf45-4ba0-a812-ab6e67966ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:52.379301 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e3aa2705-cf45-4ba0-a812-ab6e67966ef8 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:08:52.380433 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e3aa2705-cf45-4ba0-a812-ab6e67966ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:08:52.388319 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 800/1604] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:08:52.686082 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ea18-e5b0-4473-ac18-44df8d5021c8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:08:52.697139 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ea18-e5b0-4473-ac18-44df8d5021c8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:08:52.700568 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:08:52.701231 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:08:52.701488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:08:52.701825 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:08:52.711471 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:08:52.712895 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb57914-4af1-4206-867c-894dacb52f3e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:08:52.722614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 805/1605] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:09:02.374370 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:09:02.718488 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:09:05.219131 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2c98-a592-4e30-9d41-c06d4a0f3e2a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:05.219649 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2c98-a592-4e30-9d41-c06d4a0f3e2a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:05.219745 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2c98-a592-4e30-9d41-c06d4a0f3e2a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:05.221471 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 801/1606] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:09:05.226614 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ee87-efaa-4173-9f5e-53a3d001364f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:05.227108 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ee87-efaa-4173-9f5e-53a3d001364f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:05.227214 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ee87-efaa-4173-9f5e-53a3d001364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:05.245358 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8073ee87-efaa-4173-9f5e-53a3d001364f None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:09:05.246571 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8073ee87-efaa-4173-9f5e-53a3d001364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:09:05.255049 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 806/1607] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:09:05.555000 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2c98-a592-4e30-9d41-c06d4a0f3e2a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:09:05.565502 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2c98-a592-4e30-9d41-c06d4a0f3e2a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:09:05.569017 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:09:05.569648 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:05.569960 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:05.570238 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:05.579704 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:09:05.581137 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0484e4-0c80-44d1-a79f-5f07f0663387 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:09:05.591456 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 802/1608] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:09:07.653263 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8905-625e-48ca-a86c-c7b3ff4dddc1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:07.653860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8905-625e-48ca-a86c-c7b3ff4dddc1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:07.653860 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8905-625e-48ca-a86c-c7b3ff4dddc1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:07.655297 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 807/1609] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:09:07.661066 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:07.661483 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:07.661631 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:07.679711 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] MFA Rules not processed for user `f4c463b509f1490f91e3e253190d2a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:09:07.680904 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:09:07.690810 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 803/1610] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 12:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:09:07.992869 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8905-625e-48ca-a86c-c7b3ff4dddc1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:09:08.003553 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8905-625e-48ca-a86c-c7b3ff4dddc1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:09:08.007046 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf7137aab9d4ebb8b83677cc4b0d817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:09:08.007666 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:09:08.007953 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:09:08.008230 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:09:08.017043 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:09:08.019035 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e452a5a-4e22-41bd-94e3-69701302de29 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:09:08.028417 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 808/1611] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:09:17.674174 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=9 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:09:18.024281 dv-cinder-849325-1-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:11:42.242808 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:11:42.254766 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b25cf-b62f-4afd-922b-b5c9f9fd30b4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:11:42.259861 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6db07035b4d472ebc58509502b666ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2af7f192c8d41dfac1733b0da80cd73', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:11:42.260508 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:11:42.260805 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:11:42.261093 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:11:42.269601 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:11:42.271223 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833b12-6663-4998-bb84-bc8d5530bd1d service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:11:42.281343 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 804/1612] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 11 12:11:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:11:52.277107 dv-cinder-849325-1-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}